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ABSTRACT 



This document summarizes the content and conclusions of a 
workshop focusing on the transformation of traditional science, mathematics, 
engineering, and technology (SME&T) lectures and laboratories into more 
active learning environments . Presenters described innovative undergraduate 
courses in a range of SME&T disciplines. Using information technology (IT), 
these courses have been transformed in ways that appear to enhance learning 
for a diverse spectrum of undergraduate students, but workshop participants 
noted that the full educational potential of IT has not yet been realized. 
Several factors, including the difficulty of assessing student learning in 
technology- rich environments, the state of current technology, and cultural 
and institutional factors could pose barriers to rapid deployment of 
technology in SME&T classrooms. Many workshop participants thought that it is 
both possible and essential to begin planning for what the future might hold. 
The first section of the report, Innovations in Pedagogy and Technology, 
presents case studies of innovative courses and discusses evaluation and 
assessment challenges and cultural and institutional constraints to IT use. 
The second section, Planning for Uncertainty, summarizes steps that should be 
taken to make the best use of IT for future course development and the 
improvement of SME&T education. Four appendixes contain the background paper 
sent to participants before the workshop, the agenda, a list of participants, 
and biosketches of planning group members. (Contains 5 tables, 1 figure, and 
58 references.) (SLD) 
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Preface 



Information Technology (IT) enables 
exciting new approaches to undergraduate 
science, mathematics, engineering, and 
technology (SME&T) education. Cogni- 
tive research has begun to illuminate how 
students leam (National Research Council 
[NRC], 1999a) providing a basis for design 
of more effective learning environments and 
teaching practices (NRC, 1999b). At the 
same time, personal computers with 
Internet access, as well as other IT tools, are 
becoming ubiquitous on many college and 
university campuses (Web- Based Education 
Commission, 2000). Encouraged by these 
developments, small but growing numbers of 
faculty are transforming traditional SME&T 
lectures and laboratories into more active 
learning environments that hold the prom- 
ise of enhancing undergraduate learning. 

Scientists, policy makers, and research- 
ers discussed these developments at a work- 
shop, held at the National Academy of Sci- 
ences in June 2000. Presenters described 
innovative undergraduate courses in a range 



of SME&T disciplines. Using IT, these 
courses have been transformed in ways that 
appear to enhance learning for a diverse 
spectrum of undergraduate students. How- 
ever, workshop participants noted that the 
full educational potential of IT has not yet 
been realized. Several factors, including the 
difficulty of assessing student learning in 
technology-rich environments, the state of 
current technology, and cultural and institu- 
tional factors could pose barriers to rapid 
deployment of technology in SME&T class- 
rooms. 

The breathtaking pace of change in IT 
makes it virtually impossible to accurately 
predict its future impact on teaching and 
learning in undergraduate SME&T educa- 
tion. Nevertheless, many workshop partici- 
pants felt that it was not only possible, but 
also essential, to begin planning for what 
the future might hold. Some presenters 
identified steps that could be taken to speed 
development of the educational potential of 



IT and support a promising future for 
SME&.T education. 

This workshop evolved from a planning 
process begun in early 2000, when the Na- 
tional Academies created a new Center for 
Education (CFE). The new Center includes 
the Academies’ Center for Science, Math- 
ematics, and Engineering Education 
(CSMEE), Board on Testing and Assess- 
ment, and Board on International Compara- 
tive Studies in Education. To define direc- 
tions and priorities for future studies, CFE 
established a Strategic Planning Advisory 
Group. This advisory group gave high prior- 
ity to the Center’s undertaking studies of 
the impact of IT in education. As the first 
step toward such studies, the CFE convened 
a planning group to develop a workshop on 
the role of IT in undergraduate SME&.T 
education. 

The goal of the workshop was to inform 
the Strategic Planning Advisory Group, 
workshop participants, and the public about 
some issues surrounding the use of IT in 
education. To reach this goal, the planning 
group invited workshop presenters to pay 
particular attention to the following issues: 

• What educational technologies cur- 
rently exist, and how they are being used to 



transform undergraduate SME&.T educa- 
tion; 

• What is known about the potential 
future impact of information technology on 
teaching and learning at the undergraduate 
level; 

• How to evaluate the impact of IT on 
teaching and learning; and 

• What the future might hold. 

The planning group identified topics and 
speakers for the workshop, and developed 
the agenda, but did not participate in writ- 
ing this summary. Both the agenda and this 
summary build on earlier research into the 
role of IT in pedagogy and learning, pro- 
duced by CSMEE’s Committee on Informa- 
tion Technology. That committee analyzed 
and synthesized research about the ability of 
IT to enable new educational approaches, 
and identified many examples of such ap- 
proaches. This synthesis of earlier research 
was sent to all workshop participants as 
background reading, providing a broader 
context for the presentations at the work- 
shop, which focused on a few selected case 
studies. It is included in this workshop sum- 
mary as Appendix A. 
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Innovations in Pedagogy and Technology 



I hear and I forget; I see and I remember; I do 
and I understand. 

— ancient Chinese Proverb 



THE ROLE OF TECHNOLOGY 

Marshall Smith, chair of the workshop 
planning group (and an officer of the Will- 
iam and Flora Hewlett Foundation), opened 
the workshop by inviting participants “to 
look deeply at the issues of improving teach- 
ing and learning.” Smith’s emphasis on 
pedagogy was reflected in the workshop 
presentations and discussions. Although 
workshop participants believed that infor- 
mation technology (IT) has great potential 
to support improved science, mathematics, 
engineering, and technology (SME&T) 
education, most agreed that there was noth- 
ing inherent in new technologies, by them- 
selves, that would determine improvement. 
The participants noted that when IT is used 
for administrative purposes (for example, an 
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instructor posts the course syllabus on the 
Internet), it is unlikely to help students 
understand and master scientific and tech- 
nical subjects. Presenters and discussants 
focused instead on using IT to enable inno- 
vations in pedagogy that can increase learn- 
ing. Although such innovations are pos- 
sible without technology, the capabilities of 
IT make them easier and more practical. 
The new approaches to teaching and learn- 
ing discussed at the workshop reflect devel- 
opments in SME&.T education, cognitive 
science, and educational research. 

Traditionally, SME&.T courses at U.S. 
colleges and universities have been com- 
prised of lectures and laboratory sessions. 
However, a growing body of research indi- 
cates that this traditional approach is not 
effective for all undergraduates. Cognitive 
scientists have found that students have 
different ways of learning and benefit from 
different educational approaches (National 
Research Council [NRC], 1999a). In 1984, 
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Godleski argued that lectures best serve 
students who are intuitive rather than sen- 
sory learners, and a decade later, 

McDermott, Shaffer, and Somers (1994) 
found that standard physics lectures do not 
help most students grasp fundamental con- 
cepts. Researchers have also found that 
when SME&tT instructors recognize that 
student learning strategies vary and modify 
instruction accordingly, more students are 
able to learn and master these complex 
disciplines (Felder, 1993, 1996; Tobias, 
1992). 

Laboratory sessions, too, do not help all 
students develop a deep understanding of 
SME&tT concepts. Students can perform 
the experiments carefully, achieve the pre- 
determined outcome, answer questions, and 
complete the lab report, yet still leave with 
very little understanding of the concepts 
they were supposed to learn (Poole and 
Kidder, 1996). Even in some SME&tT 
courses based on newer curricula, laboratory 
experiences may emphasize verifying estab- 
lished knowledge and may not correlate 
with material presented in subsequent lec- 
tures (Hilosky, Sutman, and Schmuckler, 
1998). 

Data on course completion also indicate 
that many students cannot master SME&tT 
subjects as they are currently taught 
(Seymour and Hewitt, 1997). Among U.S. 
students who declared science and engineer- 
ing majors as freshmen in 1989/1990, fewer 
than half had completed such a degree 5 
years later, and about 22 percent had 
dropped out altogether (National Science 
Foundation [NSF], 2000). Among non- 



Asian minority students who planned to 
major in SME&T disciplines as freshmen in 
1989/1990, only 25 percent had completed 
a science or engineering degree after 5 years 
(NSF, 2000). 

In response to these problems, some 
SME&T educators are experimenting with 
innovative pedagogical methods. Often, 
these new approaches are based on research 
into human cognition, which has identified 
four elements that are key to enhancing 
learning (NRC, 1999b, pp. 19-22): 

1. Schools and classrooms must be 
learner centered. 

2. Attention must be given to what is 
taught (information, subject matter), why it 
is taught (understanding), and what compe- 
tence or mastery looks like. 

3. Formative assessments — ongoing 
assessments designed to make students’ 
thinking visible to both teachers and stu- 
dents — are essential. 

4. Learning is influenced in fundamen- 
tal ways by the context in which it takes 
place. A community-centered approach 
requires that students, teachers, and others 
share norms that value learning and high 
standards. 

Workshop participants identified several 
key elements of this new pedagogy. First, 
instructors who use technology to imple- 
ment these new approaches to teaching and 
learning typically move from lecturing, as a 
“sage on the stage,” to becoming a “guide on 
the side.” Although the instructor still plays 
a critically important role, deciding how 
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and when to intervene to enable and sup- 
port learning, the focus of the educational 
process shifts from the instructor towards 
the learner. Second, in the new paradigm, 
education can become less abstract, as both 
students and faculty can create projects or 
perform research that is useful to employers, 
community groups, or others outside the 
classroom. Third, competition to succeed as 
an individual student or professor at the 
expense of others is reduced, and collabora- 
tion becomes an increasingly powerful 
learning tool. 

Harvard University professor Christo- 
pher Dede emphasized the importance of 
this new approach to pedagogy in his pre- 
sentation on the educational potential of 
multiple interactive media. Dede noted 
that, until recently, many educators viewed 
IT primarily as a way to increase student 
access and provide economies of scale for 
traditional modes of education. 1 However, 
according to Dede, there is a “new ratio- 
nale” for using IT in higher education, be- 
yond simply increasing access. Now, more 
college and university faculty recognize that 
IT has the power to transform education by 
supporting shared creation, collaboration, 
and mastery of knowledge (Dede, 2000; see 



1 This point was reinforced during the workshop by 
Professor Richard Larson of MIT. Larson described 
the growth of private, Web-based education provid- 
ers, and of partnerships between colleges and univer- 
sities and these private vendors. Larson said that 
students anywhere around the world can now access 
upper-level MIT mathematics course materials at any 
time of the day or night. 







also Hanna et al., 2000). Like the growing 
numbers of scientists who collaborate simul- 
taneously with local and distant colleagues 
in virtual “knowledge networks,” students 
can use IT to collaborate and to learn. 

Information Technology can allow 
teaching and learning to be transformed in 
these ways, even when students do not in- 
teract with each other face-to-face, accord- 
ing to Dede. He noted that some students 
who are silent and passive in face-to-face 
settings “find their voice” and become ac- 
tive participants in technology-mediated 
communication. Using both synchronous 
and asynchronous media is important in 
producing this effect across the full range of 
learners. However, Dede argued that effec- 
tive design of learning environments will 
include careful attention to the role of fac- 
ulty mentors, as well as to technology. 
Summarizing his belief that IT is powerful 
“only if the medium is used well,” Dede 
stated “in the pedagogy lies the power.” 

To illustrate the importance of using IT 
to support this new pedagogy, rather than to 
simply expand delivery of current educa- 
tional approaches, Ben Shneiderman of the 
University of Maryland (UMD) suggested 
using new words. He proposed replacing 
“Information” in “Information Technology” 
with “Communication,” reflecting the 
power of IT to support collaborative learn- 
ing. Several recent comparative studies 
support Shneiderman’s suggestion that IT 
be used to enhance communication and 
collaboration among SME&T students. 

One recent study (Johnson, Johnson, and 
Smith, 1998) found that cooperative learn- 
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ing promotes higher individual achievement 
than either competitive or individualistic 
approaches to learning. Springer, Stanne, 
and Donovan (1997) analyzed comparative 
studies of small-group and individual educa- 
tion, concluding that collaborative groups 
promoted “greater academic achievement, 
more favorable attitudes toward learning, 
and increased persistence through SME&.T 
courses and programs.” Another meta- 
analysis (Johnson et al., 1998) reached simi- 
lar conclusions. Studies of students in biol- 
ogy (Watson and Marshall, 1995), 
chemistry (Wright, 1996), earth science 
(Macdonald and Korinek, 1995), and phys- 
ics (Hake, 1998) all indicated that students 
collaborating in small groups learned more 
than those working independently. 

Shneiderman also suggested replacing 
the “Technology” in IT with “Philosophy,” 
in keeping with his view that teachers need 
a guiding philosophy in order to meet the 



many goals of undergraduate education 
(Shneiderman, 1998b). Thus, rather than a 
technology to transmit information, IT 
becomes a “philosophy” or pedagogy to en- 
hance human communication and learning. 

T aken together, the views of those at- 
tending the workshop suggest that IT may 
make it possible to transform undergraduate 
SME&T education along the lines shown in 
Table 1-1. 

This new pedagogy reflects current 
thinking (although not yet current practice) 
in SME&T education. It is similar to the 
new educational approaches in primary and 
secondary schools called for in national 
standards for science (NRC, 1996), math- 
ematics (National Council of Teachers of 
Mathematics, 2000), and technology (Inter- 
national T echnology Education Associa- 
tion, 2000) education. 

When discussing current technological 
tools for education, presenters and partici- 



TABLE 1-1 How IT Might Transform Undergraduate SME&T Education 



Traditional SME&T Paradigm New Pedagogical Paradigm 



Teacher centered 

Instructor delivers information (information may 
be transmitted via IT) 

Individualistic (students work on their own, 
faculty work as individuals within a single 
discipline) 

Abstract 

Academia is an “ivory tower," remote from the 
outside world 

Information Technology transmits information 



Learner centered 

Students and faculty engage in active 
learning/problem-solving together 
Collaborative (students work in groups; 
faculty collaborate with other faculty 
within and across disciplines) 

Practical 

Students create service projects, useful to 
businesses, the university, or the 
community 

- Information Technology enables 
communication and learning 
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pants expressed varying views about their 
permanence and usefulness. Some agreed 
with Andries van Dam’s assertion that 
today’s Web-based educational offerings are 
“strictly transitional,” while others argued 
that in some current applications, IT is in- 
deed effectively enhancing SME&T educa- 
tion. However, these disagreements were 
quickly put aside when participants recalled 
their focus on innovations in pedagogy, 
rather than in technology. Rensselaer Poly- 
technic Institute professor Jack Wilson, who 
was a member of the earlier Committee on 
Information Technology, emphasized the 
importance of saying “loud and clear” that it 
is time “to focus on the learner and to build 
our education systems based on the best 
research.” 

Whatever their views on current educa- 
tional technologies for undergraduate learn- 
ing, participants agreed that such technolo- 
gies were likely to be very different, and 
possibly much better in the future. They 
noted that, based on the predictions and 
previous application of Moore’s Law, 2 IT 
promises to continue its rapid evolution. 



2 According to the Webopedia (www.webopedia. 
com [10/2/01]): “The observation made in 1965 by 
Gordon Moore, co-founder of Intel, that the number 
of transistors per square inch on integrated circuits 
had doubled every year since the integrated circuit 
was invented. Moore predicted that this trend would 
continue for the foreseeable future. In subsequent 
years, the pace has slowed, but data density has 
doubled approximately every 18 months, and this is 
the current working definition of Moore’s Law. Most 
experts, including Moore himself, expect Moore’s 
Law to hold for at least another two decades.” 



However, many felt that changes in colleges 
and universities, as well as increased re- 
search and development, would be required 
to capture the true potential of IT to reform 
pedagogy and enhance SME&T learning 
(see Chapter 2). 

CASE STUDIES OF 
INNOVATIVE COURSES 

Professors Jean-Pierre Bayard, California 
State University, Sacramento (CSUS), and 
Ben Shneiderman, UMD, described four 
innovative, technology-intensive SME&T 
courses. They indicated that in each of 
these courses, currently available educa- 
tional technology supported new teaching 
methods and enhanced undergraduate 
learning. The presenters’ descriptions of 
these courses also shed light on factors that 
can motivate faculty and administrators to 
try new approaches to technology and edu^ 
cation. Among the four courses, two came 
about in response to an ongoing problem of 
high dropouts and failure rates among stu- 
dents taking introductory SME&T courses. 
Faculty members developed the other two 
courses after observing that students who 
appeared successful in upper-level classes 
did not perform well afterwards, in the 
workplace and in subsequent classes. Both 
types of problems proved to be powerful 
motivators, encouraging individuals and 
groups of faculty to develop and deliver the 
innovative courses. 

At the workshop, Bayard explained that 
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he had learned about innovations in peda- 
gogy and technology while working as a 
fellow at the University of Wisconsin’s Na- 
tional Institute for Science Education 
(NISE). NSF provided funding to Bayard 
and other members of the NISE “College 
Level One T earn” to study and recommend 
improvements in undergraduate SME&T 
education, with the goal of retaining more 
students in these disciplines. Bayard noted 
that the College Level One Team initially 
focused on introductory courses, “what we 
call ‘pressure points’ and courses that... turn 
off students from science, engineering, math 
and technology.” 

Bayard said that the team used several 
criteria to select exemplary applications of 
IT that appeared to help engage and retain 
students in SME&T education. They 
sought applications of currently available 
technology (that could easily be adopted by 
other colleges and universities) across a 
variety of different types of undergraduate 
institutions. They also hoped to demon- 
strate that IT could help individual faculty 
members, departments, or institutions solve 
local problems. Finally, Bayard noted that 
they selected only those courses and pro- 
grams in which IT was “transformative,” 
helping to achieve at least one of seven 
principles for good practice in undergradu- 
ate education outlined by Chickering and 
Gamson ( 1 987 ): 3 



3 A more recent study identified a similar list of 
factors affecting college success: student-student 
interaction, student-faculty interaction, and time on 
task (Astin et al., 1993). 



• Encouraging student-faculty contact 

• Encouraging cooperation among 
students 

• Encouraging active learning 

• Giving prompt feedback 

• Emphasizing time on task 

• Communicating high expectations 

• Respecting diverse talents and ways 
of learning 

Based on these criteria, the College 
Level One team identified nine courses, 
including not only introductory “pressure 
points,” but also upper-level classes. They 
conducted detailed case studies of these 
applications of IT to SME&T education. 
Three of these case studies are summarized 
here, and all nine can be viewed at the Col- 
lege Level One “Learning through Technol- 
ogy” Web site (Millar et al., 2001). 

Bayard presented two examples of inno- 
vative SME&T courses designed to increase 
retention and learning among first-year 
students. The first was an introductory 
college algebra course at the University of 
Houston-Downtown (UHD). This urban 
university has an open admission policy and 
a diverse student body. About one-third of 
the students are African American, one- 
third are Hispanic, and one-third are of 
other races. Many students have had nega- 
tive experiences with mathematics in high 
school and lack basic skills. Although the 
university has offered a variety of remedial 
programs, seniors graduating in the early 
1990s were overwhelmingly white. The 
lack of minority students successfully com- 
pleting courses and moving on to graduate 



suggests that greater efforts are needed to 
help all students succeed (Millar et al., 
2001 ). 

The catalyst for change at UHD was the 
high failure rate from introductory college 
algebra, averaging about 70 percent across 
all sections and instructors. Among those 
students who did pass and go on to take 
calculus, instructors found that few under- 
stood the core concept of a function. As a 
result of these problems, university faculty 
did not like being assigned to teach the 
class. At that time, most mathematics fac- 
ulty followed a standard curriculum, which 
outlined specific mathematical skills to be 
taught in the introductory linear algebra 
class. For example, students were expected 
to learn how to solve a three-by-three sys- 
tem of equations. Instructors presented 
students with detailed step-by-step ap- 
proaches to solving such equations, as well 
as other types of problems included in the 
curriculum. Faculty developed and pre- 



sented example problems, theorems, and 
proofs. 

A group of mathematics instructors 
decided to try a new approach, emphasizing 
real-world problems, teamwork, and tech- 
nology. These instructors introduced graph- 
ing calculators, both to help students visual- 
ize the concept of a mathematical function 
and to eliminate some of the tedious calcu- 
lations. Today, students in some sections of 
introductory college algebra use this new 
approach, working with graphing calculators 
in small groups, where they discuss and 
solve problems. Bayard presented data com- 
paring the passing rate (a grade of C or 
higher) in these “reformed” sections of col- 
lege algebra with the rate in traditional 
sections, and the relative performance of 
the two groups in their next course (see 
Table 1-2). 

Bayard emphasized that, although these 
quantitative data reveal only a modest in- 
crease in student success as a result of the 



TABLE 1-2 Outcomes of Reforms in College Algebra at University of Houston-Downtown 



Semester 


Sections/Enrollment 


Grade C or Better 


Grade C or Better in Next Course 






Traditional 


Reformed 


Traditional 


Reformed 


Fall 1996 


Traditional: 3/75 
Reformed: 6/202 


38% 


46% 


35% 


36% 


Spring 1997 


Traditional: 3/91 
Reformed: 3/75 


38% 


39% 


32% 


26% 


Fall 1997 


Traditional: 3/74 
Reformed: 3/5 1 


45% 


48% 


20% 


29% 


Spring 1998 


Traditional: 3/87 
Reformed: 3/69 


35% 


51% 


19% 


28% 



SOURCE: Bayard. 2000. 
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changes enabled by IT, more qualitative 
indicators do indicate a large change. For 
example, he said that students “feel a huge 
difference” in terms of their interest in alge- 
bra, instructors are more enthusiastic about 
teaching, and attendance is up. Bill 
Waller, one of the instructors who devel- 
oped and teaches the innovative course, 
puts it this way (Millar et al., 2001): 

I can only give you anecdotal evidence, but 
the ambiance in the classroom is totally differ- 
ent compared to before. The biggest thing 
that I’ve enjoyed about it is when you go into 
the classroom, you’re not dreading going. 

You’re not thinking, How many are going to 
show up today? How many are going to be 
paying attention? Are they going to ask any 
questions? Am I going to get people to ask 
any questions? It’s just a much different class- 
room atmosphere than before (University of 
Houston-Downtown, “Evidence of Success,” 

p. 2). 

Bayard’s second example of innovative 
introductory SME&T education for first- 
year students was a course called “IM- 
PULSE” (Integrated Math, Physics Under- 
graduate Laboratory Science, and 
Engineering) at the University of Massachu- 
setts, Dartmouth (UMASSD). Because this 
university was formerly a technical institute, 
the mathematics and physics departments 
were happy to work with engineering faculty 
to create an integrated approach to teaching 
these subjects. 

As at UHD, the catalyst for change was 
high failure rates, but in this case, the stu- 
dent population was quite different. Engi- 
neering students who were accepted on the 
basis of their high scores on the mathemat- 
ics portion of the SAT frequently failed 
courses or dropped out altogether during 



their first year. According to Bayard, first- 
year students saw their required mathemat- 
ics, physics, and chemistry courses as irrel- 
evant — an obstacle they had to get through 
in order to reach the engineering courses 
they were interested in. Among those who 
succeeded, some found their second-year 
engineering courses were not what they had 
expected. 

To overcome this problem, engage stu- 
dents, and retain them, an interdisciplinary 
team of faculty developed the IMPULSE 
curriculum. Students enrolled in IMPULSE 
attend studio classrooms, where they spend 
most of their time working collaboratively 
in teams of three to four on a variety of 
laboratory projects. For example, each team 
may be assigned to conduct and videotape a 
physics experiment demonstrating the mo- 
tion of a projectile across the room. Using 
technology, students are able to capture 
data (such as data on the position of the 
projectile at various times) and record it in 
personal computers. Then, each team uses a 
computer to plot and analyze the data, an- 
swering questions about physical concepts 
such as velocity and acceleration. 

These projects help students learn across 
disciplines. Conducting physics experi- 
ments helps them understand concepts in 
this field. At the same time, to successfully 
analyze the data from the physics experi- 
ments, students must apply concepts of basic 
calculus, such as derivatives. And, because 
students often must troubleshoot the equip- 
ment, they develop engineering skills, in- 
cluding the ability to localize problems and 
develop alternative solutions. 
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To support the understanding of these 
concepts developed in the studio class- 
rooms, IMPULSE also includes mini-lec- 
tures in physics and calculus. The overall 
approach is borrowed from the successful 
“Workshop Physics” program at Dickinson 
College (Laws, 1997; Laws et al., 2001) and 
“Studio Physics” at Rensselaer Polytechnic 
Institute (Large, 2001). 

Collaboration is central to the IM- 
PULSE program. Engineering students 
work together in project teams, live to- 
gether in assigned areas of the dormitory, 
and take many classes together. According 
to Bayard, students objected strongly to 
being forced to live with one another, and 
also said they found the work very hard, but 
nevertheless felt that the group approach 
“helps them a lot.” 

The IMPULSE program appears to have 
met its goal of reducing attrition. During 
the fall 1999 semester, students enrolled in 
IMPULSE earned an average of nearly 16 
credit hours, or 5 credits more than the 
average among first-year engineering stu- 
dents enrolled in traditional mathematics, 
physics, and science classes in the fall of 
1998. In addition, higher percentages of 
IMPULSE students successfully completed 
calculus and physics classes on schedule 
than had students in previous years, before 
the integrated approach was adopted (see 
Figure 1-1). Overall, attrition among first- 
year engineering students has dropped from 
40 percent to 17.3 percent since the imple- 
mentation of the new program (Millar et al., 
2001 ). 

Bayard’s third example illuminates how 



O 





FIGURE I -I Percentage of first-year engineering 
students passing courses on schedule at UMASSD. 
SOURCE: Bayard, 2000. 

IT can help upper-level faculty increase 
student learning. This example was an up- 
per-level, elective geology course at San 
Diego State University (SDSU). Bayard 
said that Eric Frost, an associate professor at 
this comprehensive research and teaching 
university, became increasingly unhappy 
with his lecture-based approach in the early 
1990s. Frost observed that students who 
performed well in the class did not really 
understand the underlying concepts, based 
on their subsequent performance in other 
geology classes and in the workplace. This 
problem led Frost to dramatically change his 
approach. To match this course more 
closely to job requirements and obtain IT 
resources, he worked with an oil company 
and the San Diego Supercomputer Center 
of the University of California at San Diego. 

i b 
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Today, instead of lecturing, Frost 
teaches students to find oil and gas by learn- 
ing how to identify fault lines. He assigns 
groups of three to four students to semester- 
long projects in which they use data from 
the supercomputer to create maps. The 
governments of oil-rich nations sponsor 
some of these projects. After creating the 
maps, students search for fault lines, and 
they present their results to sponsoring part- 
ners and at scientific conferences. Frost acts 
as a “cheerleader and guide,” spending many 
hours with individuals and teams as needed. 
Although Bayard noted that there has been 
no formal evaluation of this new approach, 
he said that the oil company now provides 
scholarships to Frost’s students, hires them 
following their graduation, and moves them 
up the corporate ladder. Bayard joined 
Frost’s students for one of their regular 
weekly lunches to discuss their projects, and 
saw that they were enthusiastic and shared a 
spirit of community. 

Following Bayard’s presentation of these 
case studies, UMD computer science profes- 
sor Ben Shneiderman described his collabo- 
rative, technology-based approach to teach- 
ing upper-level undergraduate and graduate 
classes. Shneiderman explained that he was 
motivated to take this new approach during 
a visit to a large corporation that hires many 
graduates of the UMD computer science 
program. He reported that managers at this 
corporation said, “your students are great, 
but they don’t know how to work in teams.” 

This problem led Shneiderman to de- 
velop a more collaborative approach to 
computer science education over the follow- 



ing two decades. He explained that this 
collaborative approach resulted from years 
of experimenting with various elements of 
technology and pedagogy, observing which 
elements were most effective, and incorpo- 
rating those elements in subsequent courses. 
As a result of this scientific approach, 
Shneiderman said, his current teaching is 
guided by a philosophy that includes three 
components: Relate-create-donate 
(Shneiderman, 1998b). Students relate to 
each other in collaborative groups. These 
groups create ambitious projects and then 
donate them to people outside the classroom 
(Shneiderman, 2000). 

Shneiderman said that students in his 
current upper-level course, Human Factors 
in Computer and Information Systems, have 
donated a variety of projects. Some stu- 
dents in this class have helped nursing 
home directors by analyzing and reporting 
on different strategies for teaching elderly 
residents how to use computers and the 
Internet. Another team set up a database 
allowing a regional charity to track donors 
and volunteers, and another helped a local 
high school develop a plan for computer 
usage. 

As a way to encourage students in this 
course to relate to each other, Shneiderman 
has introduced “open” projects. He requires 
each student team to post its term-length 
project on the Internet and has found that 
this process encourages students to polish 
the projects and to learn more about each 
other’s work. Near the end of the semester, 
each team posts its project report, and stu- 
dents outside the project team are assigned 
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to review these draft reports. Students send 
both positive and negative comments, with 
a copy to Shneiderman. Each project team 
has three days to revise its posted report, 
based on comments received (Shneiderman, 
1998a). The final projects are posted on the 
class Web site each year and are maintained 
on the Internet as a resource for future stu- 
dents (Students of Dr. Ben Shneiderman, 
2000 ). 

Acknowledging that “I was never taught 
this stuff,” Shneiderman shared what he has 
learned about collaborative approaches to 
education. He allows students to place 
themselves on teams, based on short biogra- 
phies that they prepare at the beginning of 
each class. Students sign contracts agreeing 
to accept responsibility as team members, 
and each team is supported both by specially 
trained teaching assistants and by 
Shneiderman. 

Shneiderman argued that the open, 
collaborative approach is more successful in 
helping students master computer science 
than his past methods. He described two 
indications of this increased success. First, 
he noted that, three or four of the papers 
posted on the Web by student teams en- 
rolled in his Human Factors class in 1998 
included publishable results. One of the 
projects was accepted at an international 
conference, and a student presented it in 
Italy. By comparison, student teams en- 
rolled in earlier Human Factors classes, who 
were not required to openly post and cri- 
tique each others’ projects, produced only 
one or two projects that included some pub- 
lishable results (Shneiderman, 1998a). 



Shneiderman said that he continues to find 
that three to four in every ten undergradu- 
ate student projects can be published or 
presented in peer-reviewed venues. Second, 
Shneiderman noted that one in ten student 
projects developed currently results in an 
operational computer system that is either 
used directly or serves as the basis for imme- 
diate development of software and hard- 
ware. 

Bayard and Shneiderman provided data 
indicating that in these four selected under- 
graduate SME&.T courses, technology has 
supported new teaching approaches and 
yielded positive impacts. The anecdotal 
and quantitative data they presented indi- 
cates that these transformed courses have 
had positive effects on students, faculty, 
employers, and the community. A summary 
of the positive impacts described by these 
two presenters is found in Table 1-3. 



EVALUATION AND 
ASSESSMENT CHALLENGES 

Workshop participants agreed that it is 
very difficult to evaluate the full impact of 
IT-enabled education reforms on students, 
faculty, higher education institutions, em- 
ployers, and communities. Their views were 
similar to those participating in an earlier 
workshop on the uses of IT in undergradu- 
ate education, who observed that “assess- 
ment and evaluation of technology- 
enriched courses are in their early stages of 
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TABLE 1-3 Indicators of Positive Impacts in Case Study Courses 



Program/Course 


Institution 


Indicators of Positive Impacts 


Geological Sciences 505 


San Diego State University 


Increased student enthusiasm 
Students routinely present at conferences 
Students hired and promoted 
Graduating students offered higher salaries 


College Algebra 


University of Houston 


Improved passing rates 

Increased student motivation to study math 

Increased attendance 

Increased faculty motivation, enjoyment 


IMPULSE 

(Integrated mathematics, 
physics, engineering) 


University of Massachusetts 
Dartmouth 


Increased retention of first-year students 
Improved grades in first semester 
Students perceive they are working harder 
Students report they are learning a lot 


Human-Computer Interaction 


University of Maryland 


Increased student motivation 
More student papers accepted at 
conferences and by journals 
Provide IT services to university and 
community 

Increased teacher motivation 



SOURCE: Bayard, 2000, June; Shneiderman, 2000, June. 



development” (Ellis, Seiter, and Yulke, 

1999, p. 6). 

Workshop presenters pointed to several 
reasons for this difficulty. First, evaluations 
of educational programs are usually based on 
analysis and interpretation of data gathered 
from assessments of student learning. Yet, 
most current assessment methods have been 
designed based on a more traditional view of 
education and evaluation. Often, under- 
graduate SME&.T students are assessed us- 
ing methods that reflect only the ability to 
remember facts over a short time period. In 
a recent review of such methods, an NRC 



committee (NRC, 2001a) concluded that: 
“most widely used assessments of academic 
achievement are based on highly restrictive 
beliefs about learning and competence not 
fully in keeping with current knowledge 
about human cognition and learning” (p. 2). 

As a result, current assessment methods 
may be ill-suited to measuring student learn- 
ing gains when faculty innovate with peda- 
gogy and technology. For example, assess- 
ments that have been designed to measure 
an individual student’s achievement may be 
difficult to apply when learning is enhanced 
through small group collaboration. And, 



although a few researchers are using com- 
puters to link ongoing assessment with in- 
struction, many current assessments rely on 
paper-and-pencil methods, making them 
poorly aligned with learning enabled by IT 
(NRC, 2001a). 

A key question in evaluating the impact 
of IT in education is whether technology- 
intensive approaches provide any advantage 
in student learning, when compared with 
more traditional approaches. In theory, 
computers are capable of providing indi- 
vidualized, interactive instruction, like that 
provided by a human tutor. Because one- 
on-one tutoring has been shown to increase 
student achievement dramatically, when 
compared with one instructor teaching 20 
to 30 students, some experts argue that IT 
can yield similar gains in learning (Fletcher, 
2001). The reality, however, is that most 
current applications of IT in U.S. class- 
rooms do not take advantage of these capa- 
bilities. Researchers have conducted hun- 
dreds of studies that compare test scores and 
satisfaction of students attending classes 
where there is a live instructor with those of 
students receiving the same material via 
video, audio, or other technology-based 
delivery mechanisms. Most of these com- 
parative studies find no significant differ- 
ence in student learning with the alterna- 
tive modes of education (Russell, 2001). 

Two recent reviews of such comparative 
studies (Wisher et al., 1999; Phipps, 1999) 
found that most have been poorly designed. 
Often, investigators compared the scores of 
students who volunteered for technology- 
rich classes (and hence may be highly moti- 



vated to succeed with this delivery system) 
with students enrolled in traditional classes. 
Few studies used an experimental research 
design, with students randomly assigned to 
one of the two modes of education. An- 
other problem is that many education and 
training courses involve a mix of on-line 
and face-to-face interaction, making it diffi- 
cult to separate the effects of the two modes 
of education. As a result of such problems, 
the two reviews reached similar conclu- 
sions — that most comparative studies to 
date yield little concrete information about 
the effectiveness of IT-enabled education. 

Based on the assumption that education 
takes place when an expert teacher instructs 
students, most evaluations of the effective- 
ness of technology-rich courses have focused 
on delivery methods, rather than on con- 
tent and learning. Ben Shneiderman, not- 
ing that his computer science students like 
to conduct studies focusing on delivery 
methods, said, “I sometimes joke that it is 
like we have a running experiment to see 
whether the strawberries delivered by trucks 
or strawberries delivered by cars are fresher 
or better in some way.” 

The second, more fundamental difficulty 
in evaluating innovative applications of IT 
to transform SME&T education relates to 
goals. Workshop participants noted that 
until there is a greater consensus about the 
goals of higher SME&T education, it may 
be difficult to conduct evaluations that 
would be widely accepted. 

The first step in evaluating any program 
is to define the program’s goals. However, 
administrators, faculty, students, and par- 
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ents have varying short-term and long-term 
goals. For example, some SME&.T faculty 
may feel that, currently, the most important 
goal is to retain students within their cho- 
sen discipline. However, others may believe 
that the most important goal of undergradu- 
ate education is to create a group — possibly 
quite small — of highly qualified graduates, 
with complete mastery of specific content 
areas. This goal is reinforced by the strict 
prerequisite structures in many SME&.T 
undergraduate programs. At the workshop, 
Jean-Pierre Bayard highlighted the tension 
between this goal and the goal of retaining 
more students. He contrasted the success of 
the IMPULSE program in retaining first- 
year engineering students with his percep- 
tion that many engineering schools view 
high dropout rates among first-year students 
as a mark of success in eliminating people 
who would not become successful engineers. 

In addition to faculty and administra- 
tors, others involved in undergraduate 
SME&.T education have varying goals. 

Some students and their parents may care 
little about formal evaluations, but place 
high value on the future career success that 
might result from innovative educational 
programs. Employers may be most con- 
cerned about the ability of colleges and 
universities to prepare students to perform 
in the workplace, caring less about student 
grades or program evaluations. 

Given the problem of varying goals, 
Christopher Dede suggested applying a vari- 
ety of evaluation methods to influence dif- 
ferent audiences to adopt, develop, or de- 



mand innovative, technology-rich educa- 
tion. Many SME&.T faculty, as well as out- 
side funding agencies that support develop- 
ment of educational technology, will require 
rigorous quantitative evaluations. On the 
other hand, informal evaluations, including 
personal testimonials from faculty and ex- 
amples of student work, might help influ- 
ence some faculty to consider using new, 
technology-rich approaches. Bayard and 
others involved in the Learning through 
Technology project have gathered these 
types of informal evaluations, including 
“hallway conversations” with faculty who 
are using IT, and have made them available 
on the Internet (Millar et al., 2001). 

Workshop participants acknowledged 
that IT not only supports new educational 
approaches that may be difficult to evaluate, 
but also provides new opportunities for as- 
sessment and evaluation (NRC, 2001a). 
When students interact with educational 
software, that software can be designed to 
collect detailed records of student activities 
that provide information about student 
learning. Medical professor Ron Stevens 
(University of California at Los Angeles 
[UCLA]) presented one such approach. 
Stevens originally developed the IMMEX 
(Interactive Multimedia Exercises) program 
for medical students, and it has been modi- 
fied for use in Los Angeles-area elementary 
and secondary schools. Although IMMEX 
is not currently used in undergraduate 
SME&.T education, the program illustrates 
the potential benefits of embedding ongoing 
assessment into instruction. 
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Stevens was originally motivated to 
develop IMMEX because he saw that mul- 
tiple-choice examinations failed to fully 
assess the skills required of future medical 
doctors. Today, IMMEX is being used in 
middle schools and high schools to help 
students learn to solve problems and mea- 
sure changes in students’ problem-solving 
abilities. Extensive professional develop- 
ment is provided to teachers, who play a 
central role in helping students use the in- 
teractive software. Meeting in teams under 
the guidance of UCLA researchers, teachers 
create conceptual problems for students. 
Each problem has 50 to 60 variants, allow- 
ing students multiple opportunities to prac- 
tice solving problems that use a variety of 
types of resources and data. The software 
tracks the steps the students take in solving 
the problems and uses pattern recognition 
to categorize each student’s strategy. 
Teachers can use the tracks of individual 
student performance to assess progress and 
to compare a student’s individual progress 
with that of his or her peers. These tracks 
can also help teachers to refine instruction, 
feedback, and coaching. The information 
can also be provided directly to students, 
allowing them to think about how to im- 
prove their own performance. 

Stevens noted that a full evaluation 
of IMMEX, or any other program, should be 
multidimensional, including assessment of 
student learning, curriculum, and teaching 
practice. Chapter 2 outlines possibilities for 
such multifaceted approaches to evaluation. 



CULTURAL AND 
INSTITUTIONAL 
CONSTRAINTS 

Over the past decade, some policy 
makers, educators and members of the pub- 
lic, have questioned the quality of U.S. 
higher education. In response, individual 
colleges and universities, as well as educa- 
tional associations have launched efforts to 
improve undergraduate teaching and learn- 
ing (American Association of Universities, 
2001; Project Kaleidoscope, 1991, 1998). 
Experts have called on colleges and univer- 
sities to tap the power of IT as they under- 
take such efforts (Boyer Commission, 1998; 
NRC, 1999d). However, despite these 
reports and activities, workshop participants 
observed that transforming undergraduate 
SME&.T education with IT is a slow and 
difficult process. This is because faculty, 
administrators, students, parents, and 
employers are required to adopt new goals 
and assume unfamiliar roles (see Table 1-1). 

Workshop presenters observed that 
widespread diffusion of innovative under- 
graduate courses, such as those they de- 
scribed, might be constrained by the current 
culture and institutions of undergraduate 
SME&.T education. For example, 
Shneiderman noted that he is the only one 
among 38 professors in his computer science 
department at the UMD who uses collabo- 
rative, learner-centered approaches. Bayard 
said that other faculty at SDSU see Eric 
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Frost as unusual, and not necessarily as a 
model to be followed. Commenting on 
Frost’s new, project-based approach to 
learning, one participant noted: “I think we 
have all heard of these kinds of gems. ... 
Usually they get sealed off by the rest of the 
institution, because they say, ‘that is [effec- 
tive] because of that particular person.’ ” 

Three types of constraints — institutional 
expectations, cultural factors, and techno- 
logical constraints — could potentially pre- 
vent innovative, IT-enabled approaches 
from being implemented on a larger scale. 
First, many colleges and universities reward 
and recognize faculty based on more “tradi- 
tional” conceptions of research and teach- 
ing. Workshop participants, most of whom 
were mid- career or senior-level faculty, 
noted that they enjoyed secure, tenured 
positions, allowing them the freedom to use 
IT in new ways. Flowever, given the com- 
petitive reality of academic science, math- 
ematics, and engineering today, younger 
faculty members are motivated by, and pre- 
occupied with, the goal of obtaining tenure. 
To achieve this goal, younger faculty must 
devote so much time to research and pub- 
lishing in peer-reviewed journals that they 
may be unable to devote attention, time, 
and energy to fundamentally restructuring 
their teaching. 

Christopher Dede asserted that some 
elite universities view publishing papers in 
refereed journals as the primary measure of 
success for younger faculty. If these views 
are correct, then current institutional re- 



ward systems, emphasizing publications, 4 
may well constrain young faculty who not 
only want to improve their teaching, but 
also want to do so by using IT to transform 
their educational approach. In addition, as 
illustrated by the IMPULSE example, these 
new approaches to teaching often involve 
collaboration among faculty as well as stu- 
dents, including collaboration across aca- 
demic disciplines. Christopher Tucker 
noted that it is extremely difficult to try to 
place articles in peer-reviewed journals, and 
be awarded tenure, based on truly interdisci- 
plinary collaboration. 

Time availability may constrain faculty 
members from adopting new approaches to 
pedagogy and technology. SME&.T faculty 
who use IT to create active learning envi- 
ronments typically find themselves spending 
more time with students. In describing Eric 
Frost’s activities at SDSU, Bayard noted 
that Frost works with these students “a lot 
more than just the three hours a week that 
some of us are familiar with.” Elias Deeba, 
one of those who developed and now 
teaches the algebra course at UF1D, has said, 
“Using technology, my own workload, of 
course, has increased tremendously” (Millar 
et al., 2001). Some SME&T educators may 
be unwilling to move toward these IT-en- 
abled approaches if the increased workload 
leaves less time for research or other profes- 
sional or leisure activities. 



4 As discussed further below, some administrators 
may be reluctant to base tenure or other rewards on 
teaching quality because of a lack of good measures 
of teaching quality. 
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Institutional expectations about faculty 
time could potentially constrain develop- 
ment of technology-rich SME&.T courses, as 
well as their delivery. At UHD, faculty 
members are not expected to spend exten- 
sive time doing research, because the uni- 
versity sees its main mission as teaching. 
Reflecting this focus, faculty members are 
required to teach four courses per semester. 
This large teaching load adds to the office 
hours during which faculty are expected to 
be available for consultations with students. 
These teaching demands make it difficult to 
find time for faculty collaboration and de- 
sign of new curricula. Without the grant 
from UHD’s Teaching with Technology 
Learning Center, which offered release 
time, the professors who incorporated IT 
into the algebra course would not have been 
able to develop their new approach (Millar 
et al., 2001). 

Technological constraints could also 
slow the use of IT to transform traditional 
undergraduate SME&.T courses into more 
active learning environments. At the most 
basic level, not all colleges and universities 
have access to current technology. As illus- 
trated in the UHD example, outside funding 
is often essential to provide the hardware 
and software that students and faculty need. 
These inequities in access to technology at 
the undergraduate level are somewhat simi- 
lar to the broader problem in homes and 
elementary and secondary schools. Educa- 
tional consultant Martha Darling expressed 
her concern about the possibility of a grow- 
ing “digital divide,” between the rich, who 
have ready access to advanced technology at 



home and school, and the poor, who often 
lack such access. 

Funding to purchase and install hard- 
ware and software, by itself, will not over- 
come this problem. Student learning can be 
enabled by IT systems only when those 
systems work. For example, at the time 
Jean-Pierre Bayard visited the IMPULSE 
studio classrooms at UMASSD, tech- 
nological problems prevented students from 
videotaping the physics experiments. This 
problem illustrates the importance of having 
adequate technical support staff when 
implementing IT-enabled approaches. The 
teams of computer science students enrolled 
in Shneiderman’s classes at the UMD are 
supported not only by specially trained 
teaching assistants, but also by dedicated 
technical support staff. Many higher educa- 
tion institutions may be unable to provide 
this level of support. In addition, some 
SME&.T faculty outside computer science 
departments may lack the expertise needed 
to integrate IT into design and delivery of 
new forms of education. 

Cultural factors influence the willing- 
ness of SME&.T students and faculty to 
embrace new technology and new forms of 
pedagogy. On the positive side, undergradu- 
ates who have grown up with video games 
and home Internet access may welcome 
technology-intensive approaches to educa- 
tion. University of Arizona mathematics 
professor Deborah Hughes Hallett pointed 
out that “today’s students are powerful and 
fluent and agile in technology in ways that 
their elders are often not. This.. .is a power- 
ful boost to education.” On the other hand, 
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those students who have not had access to 
advanced tools may be at a decided disad- 
vantage in a technology- rich classroom. 

The changing approaches to education 
outlined in Table 1-1 engage students in 
active learning, often in collaborative 
groups. Although one might think that 
students would be eager to respond to these 
“learner-centered” forms of education, not 
all students are. Shneiderman noted that 
every semester, he encounters one or two 
students who are reluctant to participate in 
group projects. At UHD, mathematics in- 
structors have found that some students, 
who either dropped or failed the more tradi- 
tional sections of introductory algebra, com- 
plain that “I’ve had this course before, and 
this is not the way it’s supposed to be done” 
(Millar et al., 2001). 

Although workshop participants noted 
that all of these cultural and institutional 



constraints currently slow widespread trans- 
formation of undergraduate education, these 
constraints could possibly be reversed in the 
future. For example, high-quality, interac- 
tive software that supports individualized 
instruction and learning of SME&.T disci- 
plines might be widely available. In this 
case, a single professor could support learn- 
ing among a large, lecture-sized group of 
students, without spending hours with each 
student. In such a future, faculty and ad- 
ministrators would likely welcome the use of 
IT and the time savings that would result. 
The next chapter summarizes some partici- 
pants’ views about the future of undergradu- 
ate education, including the role of IT. The 
chapter also describes participants’ sugges- 
tions about ways to use IT more effectively 
and capture its full potential to enhance 
learning among SME&.T students. 
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Planning for Uncertainty 



A FRAMEWORK OF GIVENS 
AND UNKNOWNS 

Information technology (IT) is changing 
at a breathtaking pace, making it virtually 
impossible to accurately predict its future 
impact on teaching and learning in under- 
graduate science, mathematics, engineering, 
and technology (SME&.T) education. As 
Philip Agre of the University of California, 
Los Angeles Department of Information 
Studies put it, “everyone knows that IT is 
going to change the world, but nobody 
knows how.” Agre went on to say that, 
although no one can predict the future, it is 
possible to prepare. After discussing many 
examples of current uses of IT in under- 
graduate SME&T education on the first day 
of the workshop, during the second day, 
participants looked toward the future. To 
facilitate this process, Alan Schwartz from 



PolicyFutures 1 led workshop participants 
through the first phase of a multiple 
scenario-planning exercise, designed to 
help prepare them for future possibilities. 

Schwartz launched the exercise with 
three examples of how different organiza- 
tions ended up planning for the “wrong” 
future. His first example was the U.S. 
Army’s decision to build a large fleet of 
Abrams Ml A1 tanks in the late 1970s. De- 
signed to fight the Soviet army on the plains 
of Europe, the tanks are heavily shielded for 
protection against Soviet shells. They are 
so heavy that a military plane can carry only 
one at a time. This design, based on out- 
dated assumptions about where battles 
would be fought and what the enemies’ 
capabilities would be, has proven ill-suited 
for more recent battles. During Desert 



'A consulting firm that provides strategic plan- 
ning, scenario planning, and policy forecasting ser- 
vices to public and private clients. 
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Storm in the early 1990s, when rapid de- 
ployment was important, it took fully 6 
months to deploy the Abrams tanks. In 
addition, another aspect of the tanks’ de- 
sign — night vision that was superior to that 
of the enemy — allowed the tanks to avoid 
enemy attack during Desert Storm, making 
the heavy armor superfluous. 

A second example of misdirected plan- 
ning, based on the assumption that the fu- 
ture would resemble the past, was the cre- 
ation of the Alaska Natural Gas 
Transportation Institute in 1977, to tap the 
natural gas that is found in the oil fields on 
Alaska’s North Slope. At that time, when 
experts were predicting oil prices would 
reach $200 per barrel, parts of the pipeline 
were built, but after oil prices dropped, it 
was never completed. (With oil prices now 
rising, it is possible that the pipeline will be 
built someday; nevertheless, it is clear that 
the plans made in 1977 were not appropri- 
ate to the future that developed over the 
following two decades.) 

Schwartz’s final example was the U.S. 
automobile industry in the 1960s, which 
believed that Americans would always love 
big cars, that gasoline would continue to be 
plentiful, and that reliability wasn’t an issue 
because Americans traded in their cars ev- 
ery two years. Because U.S. auto makers 
acted based on this prevailing wisdom, Japa- 
nese firms were able to capture a large share 
of the U.S. market with smaller, more reli- 
able, and more fuel-efficient cars in the 
1980s. 

To further illustrate his point that one 
cannot reliably plan for an uncertain future 



TABLE 2-1 Changing Benchmarks 



Benchmark 


1990 


2000 


Number of Web Pages 


0 


Millions 


Dow Jones Industrial 
Average on January 3 


2.810 


1 1.358 


Soviet Union 


Exists 


Does not 






exist 



SOURCE: Schwartz. 2000. 



by simply extrapolating from current trends, 
Schwartz presented data on dramatic tech- 
nological, economic, and political changes 
that have taken place over the past decade 
(see Table 2-1). 

Schwartz suggested that an important 
step in planning for an uncertain future is to 
make a list of relevant events and to catego- 
rize the events as certain or uncertain to 
continue into the future. (For example, the 
Abrams tank was designed because the army 
felt that the existence of the Soviet Union 
was a certainty.) Schwartz maintained that 
this process of categorizing events can allow 
one to identify signposts that might be 
monitored to indicate how the future is 
unfolding. He led the workshop partici- 
pants in brainstorming to identify factors 
influencing development of technologies 
that: ( 1 ) might be useful in science and 
mathematics education, and (2) might in- 
fluence implementation of those technolo- 
gies over the next 10 years. Using the re- 
sulting list, participants then assembled in 
small groups to discuss which factors they 
felt were certain and which were uncertain. 
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Next, the small groups assembled in a ple- 
nary session, compared their lists, and cre- 
ated a framework of certain and uncertain 
factors that are likely to influence the 
course of IT in SME&.T education over the 
next decade (see Table 2-2). 

Had this session been a full multiple- 
scenario planning exercise, the participants 
would have been led to develop a frame- 
work for looking at the future based on the 
list of issues, and to identify signposts that 
might be used to monitor how the future 
unfolds. The framework and signposts could 
then be used by decision makers to build 
strategies for dealing with an uncertain fu- 
ture. Despite the lack of such a full frame- 
work, workshop participants suggested sev- 
eral strategies. Presenters and discussants 
suggested that the following steps could help 
to overcome the cultural and institutional 
constraints to change discussed in chapter 1. 

STEPS TOWARD DIFFUSION 
OF INNOVATIONS IN 
PEDAGOGY AND 
TECHNOLOGY 

Steps for Educators and Researchers 

Several workshop presenters suggested 
that the first, and most important, step that 
SME&T faculty might take would be to 
adopt a more “scientific” approach to im- 
proving SME&T education. For example, 
Shneiderman emphasized the importance of 



establishing the overriding goal of the 
course or program at the outset. Based on 
this goal, a faculty member could then for- 
mulate a hypothesis about what educational 
approaches and/or technologies are most 
likely to achieve this goal, and develop a 
protocol for evaluating progress toward the 
goal. Shneiderman suggested testing the 
hypothesis by experimenting with alterna- 
tive educational delivery methods and 
learning processes. 

Experts in the wider community, as well 
as those attending the workshop, are calling 
for a more rigorous, scientific approach to 
development of new educational approaches 
(Anderson, Greeno, Reder, and Simon, 
2000; National Research Council [NRC], 
2001b). Educators and researchers who 
adopted this approach would continually 
evaluate and refine their approach to teach- 
ing and learning. They would base their 
evaluations on assessments of student learn- 
ing gains and might also assess learning 
among faculty and instructors involved in 
innovative courses. This process would help 
educators and researchers to identify those 
educational approaches that have been 
shown to enhance student learning. Policy 
makers might focus educational funding on 
these approaches for maximum effectiveness 
(Lyon, 2001). 

At the workshop, Dede called for a new 
“culture of assessment and evaluation, 
where professors are willing to try new 
things, and view their own work critically.” 
This requires putting time and energy into 
multilayered evaluation of teaching and 
learning, including the following steps: 
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TABLE 2-2 Factors Influencing SME&T Education and IT over the Coming Decade 



Certainties Uncertainties 



Bandwidth cost 

Whether the IT tools needed to improve SME&T 
education will exist, and, if so, what they will cost 

Universal access 



Information flood 

Entertainment 

Retirement of faculty 

Demographics 

Need for learner outcomes 

Structure of universities will change 

Multiple technologies to reach millions 

Scientific developments outside the United States 

Price of educational commodities will drop 

Universities will lose monopolies 

Learning is social 

Student awareness of own skills 

Changed role of public funding 

Globalization 

Human pride 

Moore's Law 

Lifelong learning 



Student preparation for undergraduate SME&T 

Economic disparities between rich and poor 
students and institutions 

Business shifts high technology abroad 

State funding of institutes of higher education 

Whether quality mentoring will be available to all 
students 

The roles of traditional higher education 
institutions and new, for-profit education 
providers 

What does certification look like? 

Foreign reaction to U.S. dominance 

Whether technology can foster deep 
communities 

Role of assessment 

Will there be believable evidence of impact on 
outcomes? 

Economy 

Tenure policy 

Who will bear the costs of learners? One 
possibility is development of new cost-sharing 
mechanisms, such as "educational maintenance 
organizations," modeled on HMO's 

Direction of change in new institutional 
arrangements 

Cultural attitudes towards innovations in SME&T 
education 

Professional values 



ERiC 22 



32 



• Gather multiple indicators of stu- 
dent learning. Although evidence of the 
benefits of IT- enabled innovations in 
SME&T education is growing, Dede said 
that “tough” assessments are needed to dem- 
onstrate that the new educational paradigm 
really enhances student learning. These 
multiple indicators may include data gath- 
ered by the instructors of courses as well as 
data obtained by outside evaluators. 

• Assess student motivation by mea- 
suring the amount of time that students 
spend on projects, courses that they take 
over time, attendance, and retention. IT 
can help to do this, by providing logs of 
student interactions with a course or Web 
site. 

• Include both quantitative and quali- 
tative measures (which may require using 
more than one external evaluator). 

• Focus on formative assessment 2 of 
student learning and, if possible, aggregate 
to summative. Formative assessment will 
provide the information on how the project 
worked, which is essential for improvement 
and replication of the project. The Na- 
tional Institute for Science Education 
(NISE) provides resources for assessment 



2 Assessments are typically used for a variety of 
purposes. Nevertheless, many experts distinguish 
between formative assessments, used primarily to 
guide instruction and learning, and summative assess- 
ments, used primarily to determine whether a student 
has attained a certain level of competency after 
completing a particular phase of education (NRC, 
2001a). 



0 




through its Field-tested Learning Assess- 
ment Guide (FLAG) Web site (NISE, 

2001 ). 

• Match evaluation methods to the 
intended audience. For example, Martha 
Darling noted the need to demonstrate that 
the innovations “work” to end-users of the 
education system, including employers. 

• Focus assessment on students’ intel- 
lectual products and learning processes, 
rather than on technology. 

As one part of this scientific approach to 
SME&T education, faculty might collabo- 
rate across disciplines to develop new 
courses as well as to evaluate and refine 
courses they have already created. Ongoing 
assessment, evaluation, and revision of 
courses would lead to new findings about 
effective approaches. Several workshop 
participants suggested that SME&T educa- 
tors collaborate to create new interdiscipli- 
nary journals focusing on pedagogy and 
technology. The journals would provide a 
place where the results of this ongoing re- 
search could be published and widely 
shared. 

Steps for Higher Education 
Institutions 

Commenting on the innovative courses 
described by Bayard, one participant likened 
the situation in higher education to that of 
consumers and businesses faced with the 
possibility of purchasing innovative tech- 
nology. He recommended that faculty and 

— aa 
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administrators read Crossing the Chasm 
(Moore and McKenna, 1999; see also 
Rogers, 1982), which describes why high 
technology products typically are never 
widely adopted by the general public. He 
said the book argues that individuals fall 
onto a normal curve, with “early adopters” 
(who are technically literate) on one end 
and “laggards” at the opposite end. The 
problem, he said, is that individuals in the 
middle of the bell curve are unlikely to 
adopt high technology without incentives 
and rewards. 

As a first step toward bridging the 
“chasm” between early adopters of technol- 
ogy and the majority of other SME&T fac- 
ulty, workshop participants suggested that 
colleges and universities try to increase the 
majority’s knowledge about both IT and 
new approaches to education. Dede and 
other presenters identified several ways in 
which institutions might do this: 

• Help faculty learn about the poten- 
tial of collaborative, learner-centered educa- 
tional approaches. Colleges and universities 
might consider involving groups of faculty 
(both in person and in virtual learning com- 
munities) in shared learning about the new 
approaches. 

• Support cross-disciplinary collabora- 
tion to develop and implement innovative, 
IT -enabled approaches. 

• Provide financial and other support 
for student and/or faculty projects that pro- 
vide service to the university and/or to the 
community. 



However, as illustrated in Moore’s book, 
knowledge alone is unlikely to lead to wide- 
spread innovation, unless it is supported by 
recognition and concrete rewards for faculty 
who take advantage of professional develop- 
ment opportunities to restructure their 
teaching. Colleges and universities could 
publicly recognize excellent teaching that 
incorporates IT. Shneiderman noted that 
the University of Maryland recognizes inno- 
vators by inviting them to give public pre- 
sentations on successful uses of IT. Outside 
organizations, too, might grant recognition. 
One workshop participant suggested that 
the U.S. Department of Education provide 
awards and recognition for faculty who are 
exemplary in their uses of IT to enhance 
student learning. 

Several presenters and participants 
called on higher education institutions to 
evaluate and reward SME&.T faculty, based 
not only on their research, but also on the 
quality of their teaching. Some suggested 
basing tenure, salaries, sabbaticals, and 
other employment decisions on indicators of 
teaching quality, as well as research. These 
indicators might include publications in 
new journals focusing on pedagogy, peer 
review by master teachers, the number of 
publishable papers produced by students, 
and employers’ views of graduates. How- 
ever, these suggestions may be difficult to 
implement, because many faculty and 
administrators question the validity, fair- 
ness, and reliability of currently available 
methods for assessing teaching quality. 
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Steps for Public and Private 
Research and Development 

At the workshop, Christopher Dede 
presented a videotape made by a private 
firm to market its concept for a future form 
of human- computer interface. He also 
showed another vendor’s videotape, in 
which people wore wireless computing de- 
vices and interacted with “smart objects” in 
an office where computers are embedded in 
the walls and furniture. Dede noted that 
these products, envisioned by a few innova- 
tors in private firms, have the potential to 
enhance human communication and learn- 
ing. However, he said that there is a wide 
gap between these early prototypes and 
what private firms actually develop. Dede 
argued that, because most private research 
currently focuses on using IT for informa- 
tion exchange rather than for communica- 
tion, public funding is essential in order to 
develop uses of IT that enhance learning. 

Workshop participants emphasized that 
the National Science Foundation (NSF) has 
played an essential role in supporting and 
financing the kinds of innovations in 
SME&.T education that were discussed at 
the workshop. The NSF’s role has included 
analyzing, and disseminating information 
about, current innovations. For example, 
Bayard’s case studies of applying IT to un- 
dergraduate education were part of a year- 
long NSF project on “Improving Learning 
through Technology.” As one outcome of 
that project, these case studies are now eas- 
ily accessible via the project’s Web site 
(Millar et al., 2001). In 1999, NSF orga- 



nized a workshop to share experiences and 
define future priorities for the use of IT in 
undergraduate SME&.T education (Ellis et 
al., 1999), and NSF supported the National 
Academies’ workshop summarized here. 

Research and development funding from 
the NSF also was critical to developing the 
innovative undergraduate courses discussed 
at this workshop. To cite just one example, 
in 1992, University of Houston-Downtown 
(UHD) was selected to host summer work- 
shops on the writing and use of interactive 
texts as part of a project funded by NSF and 
IBM. Through these workshops, mathemat- 
ics faculty gained access to hardware and 
software that could be used to teach algebra, 
and also learned about new approaches to 
mathematics. At about the same time, the 
NSF-supported Harvard Calculus Consor- 
tium began publishing a series of textbooks 
(e.g., Hughes Hallett and Gleason, 1999), 
which gave legitimacy to the innovative 
approaches being developed by UHD math- 
ematics faculty. Finally, in addition to this 
new knowledge and technology, NSF pro- 
vided the critical grant that supported the 
three instructors’ release time while they 
developed the new curriculum (Millar et al., 
2001 ). 

Despite such successes, several workshop 
participants wondered whether innovations 
in technology and pedagogy could spread 
more widely, across departments and institu- 
tions, without additional, and larger-scale, 
funding. Andries van Dam asserted that the 
NSF typically funds educational technology 
research in $ 100,000 grants over one or two 
years, and argued that these amounts were 
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insufficient. 3 He noted that some workshop 
participants had mentioned costs of about 
$ 1 million to produce a single course “using 
today’s technology.” Van Dam called for 
larger amounts than this, over a sustained 
period, to support the long-term research 
needed to develop and implement 
tomorrow’s technology. 

Van Dam suggested that SEMATECH, 
a consortium of semiconductor manufactur- 
ers supported by both member companies 
and federal funds, might provide a useful 
model for long-term research and develop- 
ment. SEMATECH was created as a non- 
profit organization by 14 U.S. firms in 
1987, and the following year, Congress ap- 
propriated $100 million annually for 5 years 
to the new group, matching the companies’ 
contributions. Originally organized to de- 
velop procedures for manufacturing next- 
generation computer chips, SEMATECH 
soon began to emphasize strengthening the 
smaller firms that supply the chip manufac- 
turers with equipment. Acting as a commu- 
nication channel, SEMATECH helped the 
chip makers define their needs and supplier 
companies develop the capabilities to meet 
those needs. These efforts succeeded in 
helping U.S. firms gain a larger share of the 
global computer chip market in the early 
1990s, and the federal government contin- 
ued funding SEMATECH beyond the origi- 



3 The NSF has awarded much larger grants for 
educational technology in the past, ranging up to 
nearly $1 million over two years in one case. 



nal 5 years. However, since the mid-1990s, 
the organization has chosen to eliminate all 
federal support in order to open its member- 
ship to foreign firms and increase its flexibil- 
ity (NRC, 1999c). 

Jack Wilson described a current initia- 
tive that provides smaller investments de- 
signed to help diffuse innovation based on 
currently available technology. At 
Rensselaer Polytechnic Institute (RPI), he 
and other faculty and administrators have 
begun to make institutional reforms, focus- 
ing on three key elements of undergraduate 
education: computing, communication, and 
cognition. As described by Wilson, RPI 
chose to follow a “scientific” approach, first 
establishing goals, then testing alternative 
ways to reach those goals “in a way that 
would allow you to continuously refresh and 
change.” RPI’s innovations — particularly 
replacing most large lectures, recitations, 
and laboratory sections with studio class- 
rooms — have been so successful that RPI 
received an $8.8 million grant from the Pew 
Charitable Trusts. The Pew Learning and 
Technology Program, housed at RPI, is cur- 
rently using these funds to provide financial 
and technical support to other institutions 
that wish to “undertake substantial systemic 
change.” Wilson emphasized that the new 
program does not simply provide funding, 
but also works actively with institutions to 
ensure that IT is used in ways that reduce 
costs and enhance learning (Bartscherer, 
2001 ). 

Whether through new partnerships or 
through existing methods, public and pri- 
vate funding agencies could support indi- 
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vidual faculty and higher education institu- 
tions as they take a scientific approach to 
deploying IT in undergraduate SME&T 
education. In this approach, researchers 
start by establishing the goals of the educa- 
tional program. They build assessment into 
the instructional program from the start, 
and revise curriculum and delivery methods 
on an ongoing basis, based on data about 
student learning and achievement of other 
program goals. (Examples can be found in 
NRC, 2001a.) 

Workshop participants generally agreed 
that the focus of research, development, and 
demonstration should be on student learn- 
ing and quality of instruction, rather than 
on any particular technological tools. Nev- 
ertheless, some participants called for in- 
creased investments in research and devel- 
opment of particular technologies. First, 
Christopher Tucker noted that private firms 
are rapidly deploying broadband networks 
for business and residential customers, yet 
little is known about the educational poten- 
tial of these networks. He said that the 
growing availability of broadband raises 
fundamental questions about how best to 
tap its potential to enhance learning. For 
example, in traditional approaches to educa- 
tion, an instructor (whether in person or 
online) initiates the process by presenting 
some information to the learners. Tucker 
raised the question of whether, in a broad- 
band education system, the learner would 
receive a “story line” or lecture to begin the 
learning process. He noted that another 
possibility would be to create learning sys- 
tems that would be entirely self-selected or 
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self-guided, leaving open the question of 
which approach might be more effective at 
enhancing learning. 

Second, Tucker noted that new genera- 
tions of intelligent browsers can allow fac- 
ulty and students to transcend current digi- 
tal databases which replicate academic 
disciplines. For example, the American 
Economic Association produces one such 
database, EconLit. With further research 
and development, these new browsers might 
support interdisciplinary collaboration to 
enhance faculty and student learning. 

Third, Dede noted that virtual reality tech- 
nology is moving very quickly. His research 
has established that for certain types of 
SME&T subject matter, modeling and visu- 
alization in virtual reality can be very pow- 
erful in enhancing student learning (Dede, 
Salzman, Loft in, and Sprague, 1999)* Dede 
stressed the need for funding to support 
further research that would examine how 
best to use this technology for educational 
purposes. 

CONTINUING THE DIALOGUE 

Many workshop presenters and partici- 
pants expressed a desire for ongoing collabo- 
ration with others interested in developing 
innovative, effective approaches to under- 
graduate SME&T education. They noted 
the importance of continuing to learn 
about, and from, innovations in education 
and technology in both the public and pri- 
vate sectors. Some workshop participants 
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expressed a desire to learn more about the 
ways in which particular educational tools 
(such as those listed in Appendix A) might 
support the innovative approaches to peda- 
gogy discussed at the workshop. Their con- 
cerns suggested a need to create an ongoing 
forum for exchange of information. With 
support from the U.S. Department of Educa- 
tion, the NRC Center for Education initi- 
ated such a forum in late 2000, the Improv- 
ing Learning with Information T echnology 
(ILIT) Project. Although this project is 
focused on elementary and secondary educa- 
tion, its activities and findings are likely to 
guide innovations in pedagogy and technol- 
ogy in higher education as well. 

The ILIT project is designed to create 
an ongoing community of experts in tech- 
nology, cognitive science, and education 



who are devoted to improving education 
through creative applications of information 
technology. 4 These experts will work to 
develop a path toward improving teaching 
and learning in elementary and secondary 
schools through the use of IT. The overall 
project is designed to enable educational 
decision makers to make rational and strate- 
gic decisions about how they purchase and 
use education technology. Just as 
SEMATECH helped supply firms better 
understand the needs of the chip makers to 
whom they sold, the ILIT project seeks to 
empower the education community to drive 
the development of hardware and software 
tools that meet the needs of children and 
educators alike. A final project goal is to 
help guide the federal research agenda in 
the relevant areas. 



4 Further details on the project, including contact 
information, are available at: http://www4. 
nationalacademies.org/cfe/cfe.nsf. Click on “Organi- 
zational Structure and Projects,” and on “Improving 
Learning with Information Technology.”--[July 9, 
2001 ]. 
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Workshop Background Paper : 
Excerpts from a Document Prepared by 
the Committee on Information Technology 



BACKGROUND: THE 
COMMITTEE ON 
INFORMATION TECHNOLOGY 
IN UNDERGRADUATE 
SCIENCE EDUCATION 

The Committee on Information Tech- 
nology in Undergraduate Science Education 
was organized in December, 1995 and com- 
pleted its work in November of 1999. The 
committee’s charge was to improve under- 
graduate science education through the use 
of information technology. Nicholas Turro 
of Columbia University chaired the com- 
mittee. Committee members included: 
Stephen Ehrmann, American Association 
for Higher Education; Bernard Gifford, Aca- 
demic Systems Corporation; Steven Gilbert, 
American Association for Higher Educa- 
tion; B. James Hood, University of Central 
Arkansas; Deborah Hughes Hallett, Univer- 
sity of Arizona; John Jungck, Beloit College; 
Stephen Lerman, Massachusetts Institute of 



Technology; Ronald Stevens, University of 
California at Los Angeles; and Jack Wilson, 
Rensselaer Polytechnic Institute. Commit- 
tee members who served limited terms in- 
cluded Linda Chaput, Interactive Sciences, 
(December, 1995 — October, 1996); 
Stephen Hurst, University of Illinois at 
Urbana-Champaign (February, 1996 — No- 
vember, 1999); C. Bradley Moore, Ohio 
State University (December, 1995 — April, 
1997); Dorothy Stout (February, 1996-No- 
vember, 1999); and James Whitesell, Uni- 
versity of Texas at Austin (May, 1997 — 
November, 1999). 

Nancy Devino served as Staff Officer to 
the committee from December, 1995 
through February, 1999. Jay Labov served 
as Staff Officer from March, 1999 until the 
committee completed its work in Novem- 
ber, 1999. Other staff included: Gail 
Pritchard, Research Associate; Terry 
Holmer, Project Assistant; Stacy Lucas, 
Consultant; Steven Olson, Editor; and 
James Lawson, Editor. 
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TRANSFORMING 

PEDAGOGY 



INTRODUCTION 

Information technology (IT) has dra- 
matically transformed scientific practice in 
the last 50 years. The ability to handle 
massive amounts of data, collect data com 
tinuously in extraordinarily fine detail, per- 
form millions of computations per second, 
visualize information in three and four di- 
mensions with rotations and translations, 
and simulate immensely complex phenom- 
ena with multivariate, multidimensional 
models has become commonplace. Com- 
puters are ubiquitous in scientific work- 
places, whether in offices, laboratories, or 
field settings. Electronic preparation of 
manuscripts has allowed professionals 
greater control over layout, graphics, and 
symbols, such that many now use publishers 
as distributors rather than as editors, layout 
specialists, or typesetters. The World Wide 
Web, faxes, and e-mail allow scientists, 
mathematicians, and engineers to collabo- 
rate on a daily basis over long distances and 
across many time zones. New technologies 
allow scientists to control and collect real- 
time data from distant locations with expen- 
sive, highly specialized research instruments. 

This information technology revolution 



also has changed the undergraduate learning 
environment. Students are as likely to “surf 
the Web” from their dormitory rooms or 
teaching laboratories as they are to visit a 
library. They know that a portable com- 
puter or handheld calculator can perform 
almost all of the calculations expected in 
exams or problem sets. Having used com- 
puters extensively in high school, some 
students now arrive at college already know- 
ing how to enter and analyze scientific data 
in spreadsheets and statistical graphing 
packages. Many more students who enter 
college are comfortable using these tools for 
a variety of other purposes. 

Reflecting these changes, many faculty 
members have begun to post lectures and 
reading lists on the Internet, create on-line 
discussion groups, and to use IT in other 
ways to deliver courses. However, to be 
truly effective, information technology 
needs to be embedded in instruction, not just 
provided as an additional activity to a stan- 
dard course or program. For example, in 
several science, mathematics, engineering, 
and technology (SME&tT) disciplines, pro- 
fessors and instructors are using more than 
one technological innovation to help stu- 
dents learn. Earth scientists employ geo- 
graphic information systems (GIS), global 
positioning systems (GPS), and remote 
sensing. When teaching calculus classes, 
some faculty use information technology to 
emphasize multiple representations of math- 
ematical ideas (symbolic, verbal, numerical, 
and graphical) and realistic problem solving. 
Graphing calculators and spreadsheets have 
made numerical and graphical representa- 
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BOXA-1 Expanded 
Educational Experiences 
-Advocated by Reform 
Initiatives 

Education reformers anticipate that 
^science, mathematics, engineering, 
and technology faculty will recast their 
curricula to tap the broad range of 
deep content accessible through IT. 
Such changes would reflect and en- 
% courage the learning opportunities'! 
available outside the classroom, such 
as ubiquitous access to digital libraries, 
the Internet, databases of raw informa- 
tion, GIS, GPS, simulations, and other 
<* r " IT tools. However, the goal of these 
reforms is not merely to add a topic to 
■0 the syllabus of a course or curriculum, 
but to transform the curriculum so that 
students ansvyell-prepared to enter the,^ 
SME&T professional arena or participate 
I in meaningful ways in lifelong learning^; 
* ; about: science and technology, f 



tions easily accessible to undergraduate and 
precollege students. This means that most 
students — though not yet all faculty — ex- 
pect technology to be part of a mathematics 
classroom (see Box A-l). 

However, the challenge to rethink the 
mathematics curriculum has just begun. 

The emergence of calculators that can do 
symbolic manipulation (computer algebra 
systems, or CAS) will push faculty and ad- 
ministrators to consider which topics in 
algebra and calculus they need to teach with 
pencil and paper — even though machines 
can do them — because pencil and paper 



develop understanding. In addition, faculty 
need to learn how to use computer algebra 
systems to promote understanding. Can 
they be used to illuminate topics that stu- 
dents have in the past learned by rote, with- 
out much understanding? Mathematicians, 
scientists, engineers, teachers, and educa- 
tional researchers will need to reach a con- 
sensus on such questions. 

Both the power and widespread avail- 
ability of these new technological capabili- 
ties, and recent scientific findings about 
how people learn are forcing science, math- 
ematics, and engineering faculty members to 
rethink what and how they teach. Many 
have begun to use new pedagogical prac- 
tices. The committee found that IT can 
play a particularly effective role in support- 
ing six such practices: 

• visualization 

• simulation 

• real-world problem solving 

• collaboration 

• inquiry 

• design. 

The following section describes current 
technological tools that enable and support 
each of these pedagogical practices. 

CURRENT PRACTICES 

Visualization 

Computers are powerful tools to help 
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learners visualize, simulate, and construct 
objects in the created or natural worlds. 
Improvements in technology and significant 
decreases in the price of high-end computers 
mean that molecular modeling workstations 
now can be bought for as little as $2,000 
compared to ten or more times that much 
only several years ago. Molecular modeling 
offers science educators a vehicle for helping 
students understand difficult abstract con- 
cepts. For example, students often have 
difficulty understanding the relationship 
between the macroscopic appearance of a 
substance, its symbolic representation as a 
chemical formula, and its molecular shape 
and structure as predicted by quantum me- 
chanics (Gabel, 1990). Talley (1973) docu- 
mented student difficulties in understanding 
the particulate nature of matter and pro- 
posed three-dimensional visualization to 
overcome the problem. Technology offers a 
way to move beyond static two-dimensional 
representations of three-dimensional phe- 
nomena into a format that more closely 
simulates reality. Current molecular model- 
ing software can alternate images of a given 
object at rates approaching thousands of 
times per second, creating a realistic three- 
dimensional effect. The rapid increase in 
computer processing speeds means that 
software developers now are producing 
pedagogical software that was mere fantasy 
not long ago. While the full impact and 
influence of this technology on science 
education are not yet known, preliminary 
results are encouraging. Williamson and 
Abraham (1995), for example, developed 
computer animations using molecular mod- 
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eling images. They found that students who 
used these tools developed better concep- 
tual understanding of phenomena such as 
diffusion of intermolecular forces than did 
students who received conventional instruc- 
tion. 

Molecular modeling also can improve 
the linkage between lecture and laboratory. 
Crouch, Holden, and Samet (1996) de- 
scribed an innovative instructional se- 
quence in which students learn about nu- 
cleophiles and electrophiles in a lecture, 
construct electron density maps of assigned 
compounds using molecular modeling soft- 
ware, and then use these maps to predict the 
outcome of the laboratory experiment they 
are about to undertake. 

In contrast to visualization tools that 
exploit the computational capabilities of up- 
to-date computers, a GIS uses powerful 
information databases to construct detailed, 
precise computer maps. According to the 
U.S. Geological Survey, 1 “a GIS is a com- 
puter system capable of assembling, storing, 
manipulating, and displaying geographically 
referenced information (i.e., data identified 
according to their locations ).” A GIS works 
by relating information from different 
sources for the same location. Different 
kinds of data in map form can be entered 
into a GIS, and a GIS can also convert ex- 
isting digital information into maps. There 
are numerous GIS applications in wide- 



available: http://info.er.usgs.gov/research/gis/ 

title.html. [7/25/01]. 
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spread use in many different fields, includ- 
ing: 

• Agriculture (crop monitoring; land 
use management; commodity forecasting; 
soil studies; irrigation planning; and water 
resource assessment); 

• Environment (pollution, weather, 
and climate monitoring; cause-effect studies; 
landscape assessment; conservation plan- 
ning; and biodiversity libraries); 

• Health care (asset management; 
ambulance routing and mobilization; epide- 
miological studies; and road traffic accident 
analysis); and 

• Local government (planning and 
building control; land searches; boundary 
change modeling; property and highways 
maintenance; crime analysis; natural disas- 
ter management; and police and fire service 
command and control). 

This list, while not exhaustive, illus- 
trates the range of disciplines that can be 
expanded and deepened with GIS technol- 
ogy. According to the U.S. Geological 
Survey, an active GIS market has resulted 
in lower costs and continual improvements 
in hardware and software components. 

Thus, GIS tools that only recently were 
restricted to use by professional scientists 
and engineers, are now becoming part of 
undergraduate science and engineering pro- 
grams and courses. 2 

These developments are likely to result 



2 Available: http://www.ehr.nsf.gov/EHR/DUE/ 
web/ate/atelist.htm#geos. [7/25/01]. 



BOX A-2 Examples of 
Visualization Practice 

At the University of Michigan, stu- 
dents in Movement Science 303 learn 
to analyze human movements on vid- 
eotape, transferring the images to digi- 
tal format on the computer and con- 
ducting biomechanical analyses. 

Visualizing the Changing Night Sky „ 
with SKYGLOBE at the Eastern New 
Mexico University is a shareware pro- 
gram that shows the night sky at any 
moment between 30,000 BC and 
30,000 AD. It can jump through time 
in various increments (minutes, hours, 
days, months, years, and centuries) to 
indicate how the sky changes as a 
function of time. Seasonal changes, 
retrograde motion, and conjunctions 
are easily-displayed. | r ' - 



in much wider GIS application in both the 
public and private sectors, which in turn 
will pressure higher education institutions 
to provide instruction in this kind of learn- 
ing to science and engineering students. 
This example also indicates how informa- 
tion technology costs can be shared among 
the public and private sectors to everyone’s 
benefit. 

Many other visualization tools are find- 
ing a place in the postsecondary SME&.T 
curriculum (see Box A-2). Spreadsheets 
and graphing calculators are now widely 
used for mathematical modeling. Large 
databases of all types of images are acces- 
sible to students and faculty from all over 



the world. The National Library of 
Medicine’s Visible Human Project contains 
thousands of digitized cryosection photo- 
graphs, magnetic resonance images (MRI), 
and transverse computerized tomography 
(CT) images, which together comprise ana- 
tomically correct, three-dimensional repre- 
sentations of the male and female human 
body. Although used heavily in clinical 
medicine and biomedical research, these 
images also are used widely in undergraduate 
courses. The Visible Human World Wide 
Web site 3 lists dozens of applications of the 
image data sets, most of which were devel- 
oped by and are used at colleges and univer- 
sities around the country. Other databases 
contain images taken by the Hubble Space 
Telescope of planets, stars, asteroids, and 
comets , 4 which can be downloaded for use 
in astronomy courses. Digitized video can 
help students understand the physics in- 
volved in sporting events (e.g., pole-vault- 
ing), transportation (a plane take-off), and 
nature (river flow). 

Technologies that rely on interactivity 
with the World Wide Web are still develop- 
ing and hold promise as yet another avenue 
for pedagogical change. Small Java applets 5 



3 Available: http://www.nlm.nih.gov/research/ 
v is ible/v isible_human.html. [7/25/0 1 ] . 

4 Available: http://www.jpl.nasa.gov/pictures/. [7/ 
25/01]. 

5 Java is a programming language expressly de- 
signed for use in the distributed environment of the 

Internet. It was designed to have the “look and feel” 
of the C++ language, but it is simpler to use than 
C++ and enforces a completely object-oriented view 







make it possible for a Web page user to per- 
form interactive animations, immediate 
calculations, and other simple tasks without 
having to send a user request back to the 
server. Wolfgang Christian at Davidson 
College has created a Web site with numer- 
ous Java applets for physics, which he calls 
Physlets . 6 

The array of computer-based visualiza- 
tion tools available can dramatically affect 
not just how postsecondary SME&T educa- 
tors teach their courses, but how they select 
the topics to cover. 

Simulation 

Simulations let students view multiple 
aspects of complex systems simultaneously 
or sequentially. For example, an ecosystem 
or an architectural development can be 
constructed from many components with 
various connections of material and energy 
flow. Output data can be displayed in scat- 
ter plots, histograms, pie charts, and on 
maps concurrently and can be linked so that 
all representations change simultaneously 
when variables are manipulated. Similarly, 



of programming. Java can be used to create complete 
applications that may run on a single computer or be 
distributed among servers and clients in a network. It 
can also be used to build small application modules 
(known as applets) for use as part of a Web page. 
Applets make it possible for a Web page user to 
interact with the page. (Available: http:// 
www.whatis.techtargetcom/. [7/25/01] . 

6 See http://W ebPhysics.Davidson.Edu/Applets/ 
Applets.html. [7/25/01]. 
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BOX A-3 Features and Advantages of Simulations 

■ encourage student ownership of a research or problem-solving agenda; •. 

* allow students access to large, complex data sets; 1 

■ allow students to conduct the same experiment multiple times; 

* ■ draw on multidisciplinary learning, such as mathematics, in scientific exploration; 

■ help students develop quantitative skills as they engage in successive rounds of hypoth- 

esizing and logical and numerical testing of hypotheses; £ . .V 

* require interpretation of multiple representations of data; 

-JL r ■ provide novel experiences each time the simulation is run; -i, .. > 

* ■ offer flexibility in classroom contexts (e.g., case studies, investigative labs, homework, 
>£, distance education, local context, background and talents of students and teachersj; and 

'♦fallow' the use of many tools at once. 

..&• ... r > - /■ ' •• . . • . v :• . ■ *• 

SOURCE: The BioQUEST Curriculum Consortium, available at http://bioquest.org/. 
[7/25/01], 



a transformation of the data (e.g., to log-log 
scales) can be reflected in all displays in a 
linked manner. 7 Picture a situation where 
students are collaborating with each other 
or with research scientists from remote sites 
and are able to illustrate alternative graphi- 
cal and iconic representations of the same 
data to highlight patterns in that data. The 
integration of simulations with real-world 
data allows the learner to incorporate both 
theoretical and empirical data to understand 
a complex phenomenon. 

Many simulations have been con- 
structed to help students learn long-term 
research strategies. These micro worlds 
engage students in the exploration of open- 
ended problems and complex data sets, giv- 
ing them numerous modes and opportuni- 



7 For example, see BioQUEST, available at http:// 
bioquest.org/. [7/25/01]. 



ties for data analysis, visualization, and the 
ability to explore multiple variables using 
tools from many disciplines. Simulations 
allow students to investigate contemporary 
research problems, explore “what-if’ ques- 
tions, pose new problems, and examine 
fundamental concepts and classical experi- 
ments. These learning environments are 
content- and process-rich, since these types 
of simulations are open to such diverse ma- 
nipulation for long periods of investigation. 
These elaborate simulations help students 
learn which tools are appropriate in which 
contexts and why one tool is more powerful 
than another in a given context. 

Several features of currently available 
simulation tools offer advantages to faculty 
interested in enhancing student mastery of 
scientific and technical disciplines. These 
features and advantages are listed in Box 
A-3. 

Because of these features and advan- 
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BOX A-4|iExamples of -:i< 
Simulation Practices 

The Virtual Genetics Lab at the Uni- 
versity of Pittsburgh offers a course in 
conjunction with an honors course in * 
genetics, and the NextStep-based 
simulator completely replaces lab ex- 
periments. At James Madison Ulniver- | 
sity, a four-year integrated science 
and technology (ISAT) curriculum ex- 
amines issues and scientific content in 
biotechnology, energy, environmental J j 
issues, information and knowledge 
management, and manufacturing. 
Students use the computer as a pro- 
ductivity tool, simulator, model builder, 
laboratory, and communication me- f ; 
dium to concentrate on the connec- 
tions between science and the technol- 
ogy rather than disciplinary differences. ^ 



tages, simulations probably will play an 
increasingly important role in undergrade 
ate SME&T classrooms and laboratories. 

For example, in some fields, certain experi- 
ments may be too risky for novices to per- 
form, they might be too advanced for a 
student’s laboratory skills, or the necessary 
equipment might be too costly. Another 
possibility is that conducting a real-world 
experiment would require more time than is 
available in a course (e.g., multiple crosses 
or crosses of organisms with long breeding 
cycles in a genetics experiment). Simula- 
tions are a reasonable alternative (see Box 
A-4). However, wet labs offer an indispens- 
able experience to students that often can- 
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not be replaced by the use of information 
technology. 

Simulations can provide critical support 
for education based around realistic problem 
solving, such as the programs described 
earlier. By linking to research databases, 
simulations may enhance data mining 8 and, 
in some cases, allow students to do original, 
open-ended research. Using simulations, 
students can replicate aspects of historically 
important models and classical experiments. 
They also can learn long-term strategies of 
scientific research using strategic simula- 
tions. Finally, simulations are a means of 
enhancing conceptual integration and con- 
ceptual change. 

Real-World Problem Solving 

Our society faces challenges that were 
unknown to our parents and grandparents. 
Increases in global atmospheric carbon di- 
oxide may be linked to a gradual warming of 
the planet. Run-off of fertilizers used to 
grow crops and fecal waste from livestock 
can contaminate drinking water supplies 
hundreds of miles away. The safety of the 
food supply, the advantages and disadvan- 
tages of genetic testing, and the develop- 
ment of reproductive technologies to ben- 
efit infertile couples all pose difficult 



8 Data mining is the analysis of data for relation- 
ships that have not previously been discovered. Data 
mining results can include forecasting or simply 
discovering patterns in the data that can lead to 
predictions about the future. 
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challenges to society. The solutions to 
these problems will be as complex and mul- 
tifaceted as the problems themselves and 
will draw on a wide range of expertise. 

Providing opportunities for students to 
develop their problem-solving skills is be- 
coming a high priority for postsecondary 
science and engineering educators. Faculty 
members, departments, and institutions are 
recognizing the value of constructing con- 
temporary science curricula around real- 
world problems. Institutions that require 
science credits for graduation have led the 
development of many courses for non-spe- 
cialists organized around modem scientific 
problems. In some cases a semester or year- 
long course is composed of interchangeable 
modules of different topics, such as water 
quality (e.g., the Modular Chemistry Con- 
sortium based at University of Califomia- 
Berkeley 9 ); global warming (e.g., Workshop 
Physical Science 10 ), and environmental 
decision-making (e.g., The BioQUEST 
Curriculum Consortium 11 ). Publishers have 
responded with textbooks and other educa- 
tional resources that emphasize problem 
solving, including those with a 
multidisciplinary approach (Trefil and 
Hazen, 1995) and those with a disciplinary 
emphasis (Schwartz et al., 1997). 

Focusing SME&.T courses on real-world 
problems will not only encourage faculty 



’Available: http://mc2.cchem.berkeley.edu. [7/ 
25/01]. 

10 Available: http://Physics.Dickinson.edu. [7/25/ 

01 ]. 

"Available http://Bioquest.org/. [7/25/01]. 



members to develop sophisticated and real- 
istic assignments for students, but also will 
introduce students to a variety of informa- 
tion technology tools. Information technol- 
ogy can play a critical role in helping faculty 
develop real-world problems, and in helping 
students search among many possible solu- 
tions. The Internet provides access to a 
wide range of contemporary scientific prob- 
lems and potential solutions. Students can 
tap into vast amounts of unfiltered informa- 
tion, including large databases, images of 
objects of all types and sizes, and textual 
information (some of which is available in 
print, and which may or may not be peer 
reviewed). As they seek solutions to real- 
world problems, SME&.T students can con- 
nect with one another, with their instruc- 
tors, and with practicing professionals. 
Learning from scientists, mathematicians, 
and engineers in industry and in govern- 
ment (e.g., National Aeronautics and Space 
Agency, Environmental Protection Agency) 
will be crucial, since they tend to be at the 
cutting edge of the field and know about the 
most promising solutions. These experts in 
the field can help a student understand a 
complicated, interdisciplinary scientific 
problem, in ways that were impossible only 
a few years ago (see Box A-5). 

Collaboration 

Scientific research often involves exten- 
sive collaboration among colleagues. 
Single-author papers are increasingly un- 
usual in scientific and mathematical jour- 
nals. Similarly, manufacturing relies on 
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BOXA-5 Examples of Problem Solving Practices / 

In a "collaboratory" for undergraduate research and education students at Evergreen 
State College, connect in real time to instruments at research facilities, such as national labo- 
ratories, and use data collected by these instruments to solve problems posed by the profes- 
sor on their home campus. 

In the BioCalc program at the University of Illinois, special sections of Calculus & 
Mathematica for life science students use the same computer tools as the traditional sections, 
but include problems of interest to biology students such as carbon dating, population mod- 
eling, and the kinetics of drug clearance in the body. 

Real-world problems are the focus of Physics of Energy and the Environment, a course 
for non-science majors at the University of Oregon. After learning about energy, electricity, 
and fossil fuels, students spend a week exploring the effect of human activity on the ozone 
layer and global warming-; '(Available: http://zebu.uoregon.edu/l 997/physl6l .html.). [7/ 
25/01].) Click on 1 999 Course. 

The BioQUEST Library is a compendium of computer-based tools, simulations, and textual 
materials that supports collaborative, research-like investigations in biology classrooms. For 
example. Evolve is a computer simulation that allows Sisers to model evolution experimentally 
by controlling a number of variables, including the starting population size, overall popula- 
tion size, intensity of natural selection, pattern of inheritance, and proportion of migration in 
a hypothetical population. (Available: http://www.apnet.com/bioquest/.) (7/25/0 1). 

Harvey Mudd College has a nationally recognized clinic program centered on a multi- 
disciplinary design-oriented approach to real-world problems. The computer science, engi- 
neering, mathematics, and physics departments sponsor about 40 "clinic projects" annually 
in which students work in groups of four or five under the guidance of a student team leader, 
a faculty advisor, and a' liaison .from the sponsoring organization A 1996-1997 clinic was 
sponsored by BHK Enterprises, a local Southern California supplier of lamps for scientific and 
industrial use. The BHK clinic involved research to discover and design an improved deute- 
rium plasma arc source. This clinic required students, to acquire a fundamental understand- 
ing of, and the skills to model/the mechanisms of gaseous excitation and emission, as well* as 
empirical design based on student measurements. (Available: http://www.hmc.edu/acad/ 
clinic/.) (7/25/01). 



teams of designers to create new products, as 
do software companies to write code for a 
software application. Postsecondary institu- 
tions also need to provide students the op- 
portunity to participate in collaborative 
projects in which they can develop and 
nurture their teamwork skills using modem 
information technology as a tool. 



Information technology can offer all 
students the opportunity to leam teamwork 
skills, including students who live off cam- 
pus. These students often have full-time 
jobs in addition to family responsibilities. 
Without access to information technology 
and telecommunications, they would be at a 
distinct disadvantage to their on-campus 
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BOX A-6 Examples of Collaboration Practices 

Billie's Story— An Experiment in Molecular Biology Using CD-ROM is the result of an 
experimental course at Vanderbilt's Peabody College that brings together molecular biology, 
scientific methodology, and technology. Six students worked together to develop an inter#? 
tive multimedia program that could be used in classrooms in the future They produced a 
multimedia CD-ROM called Billie's Story about a youngster with cystic fibrosis. The product 
includes extensive information on genetics, gene therapy, detection and diagnosis, treat- 
ment, and medical ethics, focused around cystic fibrosis. The program also includes links to 
World Wide Web sites that the students found useful to their research, (Available: http:// 
peabody.vanderbilt.edu.). [7/25/Of]./ North Dakota State University's Introductory Psychol- 
ogy course has a "lectureless" online instructional format that includes electronic mail and 
course Web pages. Students enrolled in the class use these tools to collaborate on group 
problem-solving exercises. Introductory astronomy students at the University of Iowa use a 
research-grade, automated telescope for laboratory exercises and research projects. The 
students use a Web-based observing form to submit observing requests, and then receive 
images in a networked directory folder the next mornirfg. ’ :V 



peers. But students need not be limited to 
virtual interactions on the World Wide 
Web. Collaborative projects also can in- 
volve more discrete team activities such as 
the development of a CD-ROM. The first 
of the three examples in Box A-6 is a 
project designed to develop both informa- 
tion technology and teamwork skills. 

Inquiry 

Recent changes in science, mathemat- 
ics, and technology education at the K-12 
level promise to have a profound impact on 
postsecondary education. National stan- 
dards in mathematics (National Council of 
Teachers of Mathematics, 1989), science 
(American Association for the Advance- 
ment of Science, 1993; National Research 
Council [NRC], 1996b), and technology 



(International Technology Education Asso- 
ciation, 2000) have been developed and are 
being adopted and translated by state de- 
partments of education. In the not-too- 
distant future, high school students will 
begin arriving at college with very different 
learning experiences and expectations 
(NRC, 1999b). 

For example, the National Science Educa- 
tion Standards (NRC, 1996b) emphasize 
inquiry as a means of learning fundamental 
scientific concepts. Inquiry is defined in the 
Standards as “a set of interrelated processes 
by which scientists and students pose ques- 
tions about the natural world and investi- 
gate phenomena” (NRC, 1996b, p. 214). 
When engaging in inquiry, students con- 
struct explanations, test those explanations 
against current scientific knowledge, and 
communicate their ideas to others. They 
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identify their assumptions, use critical and 
logical thinking, and consider alternative 
explanations. In this way, students actively 
develop an understanding of science by 
combining factual scientific knowledge with 
reasoning and thinking skills. 

College courses historically have con" 
sisted of lecture, recitation, and laboratory. 
However, this configuration, and parties 
larly the lecture, may contribute little to 
some students’ learning. Standard physics 
lectures do not help most students grasp 
fundamental concepts (McDermott, Shaffer, 
and Somers, 1994). Birk and Foster (1993) 
compared student grades in large multi" 
section lecture courses and found that the 
experience and lecturing skills of the in" 
structor did not correlate with student 
achievement in the course. Although Birk 
and Foster conceded that results could be 
attributed to a mismatch between the 
instructor’s goals and the assessments used 
to determine student grades, they main" 
tained that the lecture method is not the 
best learning tool for most students. For 
these reasons, students who have been ex" 
posed to inquiry"based learning in their K" 
12 education could be frustrated by many of 
the prevailing postsecondary SME&T peda" 
gogical practices, particularly at the intro" 
ductory level. 

Unfortunately, the laboratory compo" 
nent of many undergraduate SME&T 
courses also can be less than optimal in 
developing students’ conceptual under" 
standing. The experiments and activities in 
these courses are often structured in such a 
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way that students can perform the actions 
step"by"Step and achieve the predetermined 
outcome to within a small margin of error 
(if they are careful). Students may answer 
questions that are drawn from the prepara" 
tory information at the beginning of the 
activity and fill in the lab report at the 
end — yet still leave the laboratory with very 
little understanding of what was supposed to 
be learned (Poole and Kidder, 1996). Even 
in “newer” reforrmminded curricula, labora" 
tory experiences often are selTcontained, 
emphasize verification of established con" 
elusions, and do not correlate with material 
presented in subsequent instructional ses" 
sions (Hilosky, Sutman, and Schmuckler, 
1998). 

Inquiry as defined by the above stan" 
dards, and “extended inquiry” in which 
students undertake real scientific research 
projects extending over several days or 
weeks, present major pedagogical challenges 
to their instructors. In the process of investi" 
gating a scientific question, both students 
and educators are required to use such di" 
verse skills as the following (Tinker, 1997): 

• modes of thinking, including posing 
good questions and developing experimental 
strategies; 

• procedural skills, such as data collec" 
tion and instrument calibration; 

• familiarity with design and construe" 
tion (safety procedures or machine shop 
skills); and 

• analytical techniques, such as graph" 
ing, mathematical modeling, and statistics. 
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Although most scientists with doctoral 
degrees acquire the skills listed above during 
completion of their dissertation research, 
they may not feel qualified to teach them. 
They also may feel constrained by depart- 
ment-wide curricula and the need to cover a 
specified range of topics for students to en- 
roll in subsequent courses. Faculty time also 
can be a factor when considering the effort 
necessary to develop or modify laboratory 
activities (for which professional recogni- 
tion or rewards are often minimal). An- 
other barrier to the kinds of reform advo- 
cated by the NRC (1996a, 1999b) and the 
National Science Foundation (NSF) (1996) 
is the presence of faculty members with 
master’s degrees who have not conducted an 
extended research project. An additional 
problem is that beginning graduate students, 
who have little experience with extended 
inquiry projects, are often employed as 
teaching assistants and lead laboratory 
classes. 

An increasing number of academic insti- 
tutions are beginning to integrate computers 
into introductory science and engineering 
courses. In the traditional lecture/recita- 
tion/laboratory system, computers have 
been used as aids for demonstrations in lec- 
tures, to run simulations, in microcomputer- 
based laboratories, and for out-of-class prob- 
lem solving. This initial practice often 
encourages even more ambitious technol- 
ogy-based projects such as the use of Calcu- 
lus &. Mathematical 2 Workshop Physics 



12 Available: http://www-cm.math.uiuc.edu/. [7/ 
25/01]. 



(Laws, 1991, 1999), and Studio Physics 
(Wilson, 1994). 

Technology now provides exciting new 
opportunities for learners to conduct their 
own investigations of real problems (see Box 
A-7). In addition to working with data 
generated or collected by others, informa- 
tion technology enables students to collect 
and analyze their own scientific data with- 
out regard to time or place. Commercially 
available computer- interfaced sensors can 
measure almost any phenomenon, and they 
can be used with handheld calculators, bat- 
tery-operated computer-based laboratory 
interfaces, and personal data assistants, 
making them opportune for fieldwork. 13 In 
experiments not suited for computer-based 
data collection, technology still can allow 
learners to manipulate and analyze data, fit 
data to known mathematical equations, 
rapidly analyze large data sets with sophisti- 
cated statistical tests, simulate with models, 
and draw conclusions based on the out- 
comes of these processes. Students may 
even have access to “real” sensors used by 
scientists, such as devices for measuring the 
temperature of a lava flow or obtaining wa- 
ter samples as a function of depth. With 
modem networking, these instruments that 
collect data may be a world away from the 
students. 



13 See The Concord Consortium for further infor- 
mation about sensors and other uses of information 
technology in education. Available: http:// 
www.concord.org/. [7/25/01]. 
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BOX A-7 Examples of Inquiry Practices 



Principles of Global Dynamics at the University of North Carolina, Chapel Hilfyisaquanti- 
tative study of the earth system as a whole, from the core to the upper atmosphere. The 
course provides a global view of the planet and describes modern approaches to monitoring 
the earth, its interior dynamics, and its external environment. An important feature of the 
course is an electronic document that follows the contents of the course and allows students 
to complement and expand the information by following hyperlinks : to Web sites selected for 
the course. One section of the course teaches students explicitly how to access, filter, and 
use a wide array of electronic resource materials. Finally, a separate electronic discussion 
forum is available to students, so that they may post comments, e-mail their homework, and 
notify their classmates and instructors of interesting data or links found. Montana State Uni- 
versity offers a course fbricertified K-1 2 teachers to learn how to use the Internet to teach 
mathematics and science in the context of earth system science. This course investigates the 
relationships among changes in the atmosphere, ocean circulation patterns, and environ- 
mental processes both on and below the earth's surface. From the Internet, curricular mate- 
rials and Resources designed by "Network Montana Project Earth System Science" serve as 
model lessons to teach aspiring teachers how to integrate concepts with image processing 
and analysis techniques for their own future instructional environments. In the Concepts of 
Physics program at Kansas State University, non-specialists, including prospective elementary 
school teachers, pose questions about physical phenomena and use interactive digitized 
video to obtain answers. \ l 
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Design 

Computer-aided design (CAD), once 
exclusively the domain of professional engi- 
neers, is now used in undergraduate engi- 
neering courses as early as the freshman 
year. Although CAD bears a superficial 
resemblance to simulations — they both 
attempt to reproduce real phenomena 
graphically — CAD is only one step on the 
pathway to designing and manufacturing a 
real product. Growing numbers of engineer- 
ing schools are offering CAD instruction, 
because it provides important workplace 
skills. For example, many professional engi- 
neers now use AutoCAD, a high-quality 



design tool. Even some high school stu- 
dents are now learning to use these tools 
(Ercolano, 1998). 

As distance education and collaboration 
between individuals at different sites be- 
come more common, the use of design tools 
that can be transmitted over the Internet 
will likely increase. Designers can build a 
sequence of visual images into Web settings, 
and a user can view, move, and rotate those 
images, as in the example found in Box A-8. 14 



14 Available: http://whatis.techtargetcom/ 
vrml.htm for definition of VRML. [7/25/01]. 
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BOX A-8 Example of 
Design Practice 

f '-2 - •• ' •• 

Students in the Structural Design in 
Architecture program at the University 
of Virginia have electronic access to 
hundreds of high-quality images of 
human-constructed structures around 
yy, the world. They use digital manipula- 
te tion techniques to understand and ex- 
plain structural concepts, such as the 
y ability to remove elements from a struc- 
% tural system to illustrate its behavior- 
simulating failure modes by distorting 
images; and adding annotations to 
! explain mathematical concepts graphi- 
') ca I ly. 



APPLICATIONS OF IT ACROSS 
SME&T DISCIPLINES 

Many of the six pedagogical practices 
described above can be incorporated into 
courses that traditionally have not had a 
strong information technology component. 
In addition to the examples presented 
above, information technology is central to 
many other current efforts to improve 
teaching and learning in undergraduate 
SME&T fields. T able A- 1 provides addi- 
tional examples of innovative pedagogical 
practices, organized by discipline. 



ASSESSING STUDENT 
LEARNING 

Currently, the dearth of effective instru- 
ments for measuring the effectiveness of 
new teaching and learning approaches that 
incorporate IT poses a tremendous barrier to 
the diffusion of these new approaches. 
Evaluation instruments must allow for 
metrics that the science and engineering 
teaching community are willing to accept. 
Some educators who resist curricular change 
argue that, until standardized tests such as 
the Graduate Record Examinations and 
Medical College Admission Test change 
their orientation to emphasize analysis of 
information, the solution of open-ended 
problems, and design of experiments to test 
hypotheses, using information technology to 
transform undergraduate education cannot 
succeed. 

To make progress, pioneering faculty 
members will need to become actively in- 
volved with changing these tests. Although 
developing student learning assessments to 
correspond with new pedagogical methods 
can be extremely challenging, it can be 
done. For example, a group of chemistry 
professors at the University of Wisconsin- 
Madison felt strongly that their “structured 
active learning” (SAL) approach in intro- 
ductory chemistry resulted in increased 
student learning. However, their colleagues 
were reluctant to embrace it. Consequently, 
faculty members from the science, engineer- 
ing, and mathematics disciplines and on- 
campus experts in assessment and statistics 
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TABLE A-l Examples Highlighting Pedagogical Practice Innovation 



Discipline 

Earth and 
Space Science 

Astronomy 

Geography 



Geology 

Life Science 

Biology 



Immunology 



O 




Example 



Pedagogical Innovation 



Project CLEA 
Gettysburg College 
http://www.gettysburg.edu/project/ 
physics/clea/CLEAhome. html. 

[7/25/0 1J. 

The Geographer's Craft 
University of Texas at Austin 
http://www.colorado.edu/geography/ 
gcraft/contents/html. [7/25/01]. 

(The Geographer's Craft pages have 
moved to the University of Colorado.) 



"An Integrated, ComputerAssisted 
Approach to Teaching Introductory 
Geology Laboratories" 

University of Illinois, Urbana-Champaign 
http://www.geology.uiuc.edu/HTML 
Click on: SHurst, fieldtrips [7/25/01]. 



BioQUEST Curriculum Consortium 
Beloit College 

http://www.bioquest.org. [7/25/01 ]. 



Workshop Biology 
University of Oregon 
http://biology.uoregon.edu/Biology_ 
www/workshop_biol/wb.html. [7/25/01 j. 



IMMEX 

University of California at Los Angeles 
http://www.coled.umn.edu/edutech/ 
immex/. [8/8/01]. 



■ Visualization 

Laboratory exercises use digital data 
and color images to illustrate modern 
astronomical techniques. 



■ Visualization 

■ Real-World Problem Solving 
Students address research problems with 
"appropriate" geographical concepts 
and techniques drawn from cartography, 
geographic information systems, remote 
sensing, spatial statistics, and other 
information technologies. 

■ Visualization 

■ Simulation 

Electronic geology field trips 



■ Simulation 

■ Real-World Problem Solving 

■ Collaboration 

Publishes The BioQUEST Ubrary : a collection 
of simulations for learning long-term 
research strategies; tools, data sets, and 
modeling. 

■ Simulation 

■ Real-World Problem Solving 

■ Collaboration 

Epidemiology, demography, and cardiac 
physiology simulations published in The 
BioQUEST Ubrary. 

■ Real-World Problem Solving 
Artificial neural networks that distinguish 
novice and expert strategies during 
complex problem solving. 

continued 
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TABLE A- 1 Continued 



Discipline 

Physical Science 

Chemistry 



Example 



Pedagogical Innovation 



ChemLinks 
Beloit College 

http://chemlinks.beloit.edu/. [7/25/01 J. 

Modular Chemistry Consortium 
University of California, Berkeley 
http://mc2 . cchem . berkeley. edu/. 
[7/25/01]. 



New Traditions 
University of Wisconsin 
http://newtraditions.chem.wisc.edu/. 
[ 10 / 3 / 01 ]. 



Molecular Science Initiative 
University of California Los Angeles 
http://server2 . nsic. ucla . edu/ms/. 
[7/25/01]. 



■ Visualization 

■ Simulation 

■ Real-World Problem Solving 

■ Collaboration 

■ Inquiry 

Applications of technology include 
multimedia components for illustrating 
concepts through simple simulations, 
animations, and video clips; World Wide 
Web-based literature searching; MathCAD- 
based problem solving; molecular 
modeling; graphical representations of 
large data sets with Quick Time movies; and 
a few modules with large-scale interactive 
multimedia components. 

■ Visualization 

■ Simulation 

Curriculum project for undergraduate 
chemistry includes MathCAD-based 
interactive texts for physical chemistry; 
ChemScape, a multimedia encyclopedia of 
lab techniques to free instructor time and 
facilitate distance learning; CyberProf, an 
integrated, easy-to-use system for delivering 
course material through the Web; and 
molecular modeling exercises. 

■ Visualization 

■ Simulation 

■ Collaboration 

Development and implementation of a fully 
digital and network-deliverable molecular 
science curriculum, including WebCT for 
developing sophisticated Web-based 
environments. Calibrated Peer Review 
software. Mastering Chemistry tutorial 
software, and multimedia "Exploration" 
tools. 
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TABLE A- 1 Continued 



Discipline 



Physics 



Mathematics 



O 




Example 



Pedagogical Innovation 



Workshop Physics 
Dickinson College 

http://physics.dickinson.edu/Physics 
Pages/Workshop_Physics/Workshop_ 
Physics_Home.htm. [7/25/01 J. 



Studio Physics 

Rensselaer Polytechnic Institute 
h ttp.V/www. rpi . edu/dept/phys/studio_ 
physics/phys 1 /physlmain.html. [ 1 0/3/0 1 ]. 



The Center for Science and 
Mathematics Teaching 
Tufts University 

http://ase.tufts.edu/csmt/. [7/30/0 1 j. 



Calculus St Mathematica 

University of Illinois, Urbana-Champaign 

Ohio State University 

http://www-cm.math.uiuc.edu/. 

[1 0/3/01 J. 

Calculus, Concepts, Computers and 
Cooperative Learning (C4L) 

Purdue University 

http://www. math . purdue. edu/~ccc/. 

[1 0/3/01 J. 



■ Visualization 

■ Real-World Problem Solving 

■ Collaboration 

■ Inquiry 

Activity-based real-time data acquisition and 
analysis; design of the curriculum is 
informed by outcomes of physics education 
research. 

■ Visualization 

■ Real-World Problem Solving 

■ Collaboration 

■ Inquiry 

Computers are used for the acquisition and 
analysis of data, which can be collected 
either from sensors or digitized video. 

■ Real-World Problem Solving 

■ Collaboration 

■ Inquiry 

The Center develops curricula, activities, 
and computer tools that allow students to 
participate actively in their own learning 
and to construct scientific knowledge for 
themselves. Using these materials, the 
students learn directly from the physical 
world. The Center's substantial conceptual- 
learning research and evaluation program 
guides the development of materials. 



■ Visualization 

■ Real-World Problem Solving 
Lectureless virtual courses using a symbolic 
algebra package for calculus education in a 
computer lab context. 

■ Visualization 

■ Real-World Problem Solving 

■ Collaboration 

Use of Interactive SET Language, an 
interpreted mathematical programming 
language closely resembling the language 
of sets and functions and Mathematica for 
calculus. 

continued 
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TABLE A- 1 Continued 



Discipline 



Example 



Pedagogical Innovation 



Harvard Calculus Consortium 
Harvard University 
http: //www. math, harvard . edu/. 
( 10/3/0 1 J. 



Project CALC: Calculus as a 
Laboratory Course 
Duke University 

http://www.math.duke.edu/education/ 
proj_calc/. [10/3/01 J. 

Calculus from Graphical, Numerical, 
and Symbolic Points of View 
St. Olaf College 

http://www.stolaf.edu/people/zorn/ 
ozcalc/index.html. f 1 0/3/0 1 J. 

Workshop Mathematics 
Dickinson College 
http://www.workpage.com/ 
f/84/546f.htm. [I 0/3/01 J. 



Calculus in Context 
Smith College 

http://www.math.smith.edu/Local/ 
cicintro/cicintro.html. [7/25/01 j. 



■ Visualization 

■ Real-World Problem Solving 
Extensive use of graphing calculators for 
visualization, numeric modeling, and 
problem solving. 

■ Visualization 

■ Real-World Problem Solving 
Real applications and complex data 
modeled with computer packages such as 
MathCad, Mathematica, and Maple. 

■ Visualization 

■ Real-World Problem Solving 

Single variable and multivariable calculus 
explored with extensive use of graphics and 
computational environments. 

■ Visualization 

■ Real-World Problem Solving 

■ Collaboration 

■ Inquiry 

Introductory mathematics that takes 
advantage of recent research findings in 
mathematics and science education and 
makes effective use of new computer 
technologies. 

■ Visualization 

■ Real-World Problem Solving 
Use the method of successive 
approximations to define and solve 
problems, develop geometric visualization 
with hand-drawn and computer graphics, 
while giving numerical methods a more 
central role. 



TABLE A- 1 Continued 



Discipline 



Engineering 



Example 



Engineering Academy of New England 
University of Connecticut 
University of Massachusetts-Amherst, 
University of Massachusetts-Lowell, 
University of Rhode Island, and Hartford 
Graduate Center 
http://www.egr. uri.edu:80/ne. 

[ 1 0 / 3 / 01 ). 



The Gateway Engineering Education 
Coalition 

Columbia University, Cooper Union, 
Drexel University, New Jersey Institute 
of Technology, The Ohio State 
University, Virginia Polytechnic University, 
and University of South Carolina 
http://www-gateway.vpr.drexel.edu. 
(10/3/01). 



Southern California Coalition for 
Education in Manufacturing 
Engineering (SCCEME) 

California State University-Fullerton, 
California State University-Long Beach, 
California State University-Los Angeles, 
University of California-lrvine, 

University of California-Los Angeles, and 
University of Southern California 
http://www.csulb.edu/colleges/coe/ 
main/index.html. (10/3/01). 



Pedagogical Innovation 



■ Simulation 

■ Real-World Problem Solving 

■ Collaboration 

■ Design 

Modern communications technologies 
enable collaboration between five 
engineering and technology education 
programs and their respective industrial 
partners to produce a set of engineering 
courses, curricula, and workforce training 
programs. Information technology-based 
curriculum projects feature interactive 
multimedia tools for simulating the 
manufacturing design process. 

■ Simulation 

■ Real-World Problem Solving 

■ Collaboration 

■ Design 

Multimedia computer-aided instruction 
tools have been developed for 
introductory and intermediate materials 
engineering courses. A major thrust of 
the coalition's work is a collection of 
collaborative educational technology and 
methodology projects that facilitate 
sharing of data, information, and 
resources among member institutions. 



■ Visualization 

■ Simulation 

■ Collaboration 

Member institutions are developing a 
variety of hypermedia-based instructional 
modules covering modern methods and 
new technologies for manufacturing to 
be used in campus-based engineering 
courses and workplace-based training 
programs. This effort includes 
development of a "Simple Authoring 
Environment" for cross-platform, rapid 
development of hypermedia educational 
materials. 



were recruited to assess learning by students 
emerging from either the traditional intro- 
ductory chemistry course or the SAL course. 
The faculty members used their own criteria 
and instruments to see if they could distin- 
guish the two groups of students. All statis- 
tical measures showed that the assessors 
ranked the SAL students as more competent 
(Millar et al., 1996; Springer et al., 1997a). 
As a result of this assessment, educators who 
had previously expressed skepticism about 
the effectiveness of the new approach be- 
came more open to new teaching methods. 

As Garnett et al. ( 1995) have observed, 
when passive transmission of information is 
replaced by active learning activities, “the 
student is required to think carefully about 
the information, analyze it, and then apply 
it to new situations that may not have a 
clear solution or at first glance seem to be 
unrelated to the original presentation con- 
text.” As a result, these researchers find 
that student understanding improves — 
provided the tools used to measure it are 
consistent with what one defines as “under- 
standing.” 

As the SAL example illustrates, faculty 
members have considerable control over the 
assessment strategies they use in their own 
classes. These faculty members will need to 
consider how to improve the alignment 
between instructional goals and assessment 
tools. Tests often drive student behavior, 
especially at the classroom level. Research- 
ers, teachers, and testing specialists there- 
fore are exploring and experimenting with 
new assessment approaches, including dis- 
covering those that map more directly onto 



integrative and constructed learning. What- 
ever approach is chosen, assessment must at 
least provide feedback to teachers and stu- 
dents to make a fair judgment of progress 
and sensible next steps. 

Before describing good assessment prac- 
tices in current use and suggesting ways to 
use information technology to improve the 
process, it is important to define the various 
forms of assessment and distinguish between 
assessment and evaluation. The American 
Association for Higher Education (AAHE) 
defines assessment as “an ongoing process 
aimed at understanding and improving stu- 
dent learning.” 15 Most definitions of evalu- 
ation, on the other hand, put little emphasis 
on feedback and instead concentrate on 
discrete measures of the overall success or 
failure of a process or project. Assessment 
often has been categorized as formative or 
summative. Formative assessments provide 
feedback to the student and instructor in a 
regular and systematic way, but often are 
not part of the student’s final grade. 
Summative assessments measure student 
knowledge at the conclusion of instruction 
(whether a chapter, a part of a semester, or 
at the end of the semester), but those results 
are rarely used to improve teaching and 
learning. Although tests of various sorts are 
by far the most common form of assessment, 
other assessments that measure student 
ability to carry out a task (performance as- 
sessments) or the ability to apply learning to 



15 Available: http://www.aahe.org/assessment/ 
assess_faq.htm#define. [7/27/01]. 



the solution of real-world problems (authen- 
tic assessments) are becoming more popular. 

Principles of Good Practice in 
Assessing Student Learning 

Several national efforts are aimed at 
helping postsecondary faculty members 
reconsider assessment issues. One of these, 
the Assessment Forum of the AAHE, builds 
on the foundation laid more than a decade 
ago by Chickering and Gamson (1987). 
Although not aimed specifically at SME&T 
educators, the following recommendations 
by AAHE (1992) are flexible and relevant 
enough to encompass a wide range of educa- 
tional innovations and can frame design 
assessments of information technology- 
based innovations in the SME&T curricu- 
lum: 16 

1. The assessment of student learning 
begins with educational values. 

2. Assessment is most effective when it 
reflects an understanding of learning as 
multidimensional and integrated, and is 
revealed in performance over time. 

3. Assessment works best when the 
programs it seeks to improve have clear, 
explicitly stated purposes. 

4. Assessment requires attention to 
outcomes but also and equally to the experi- 
ences that lead to those outcomes. 

5. Assessment works best when it is 
ongoing and not episodic. 



16 Available: http://www.aahe.org/. [7/27/01]. 
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6. Assessment fosters wider improve- 
ment when representatives from across the 
educational community are involved. 

7. Assessment makes a difference when 
it begins with issues of use and illuminates 
questions people really care about. 

8. Assessment is most likely to lead to 
improvement when it is part of a larger set 
of conditions that promote change. 

9. Through assessment, educators meet 
responsibilities to students and to the pub- 
lic. 

In the current climate of accountability 
in all types of higher education institutions, 
assessment must be included as a regular 
feature of all information technology activi- 
ties. Student performance and progress will 
be key indicators for the next decade 
(Baker, 1998). Some of these assessment 
metrics can be as simple as usage and time 
on task. Others can be more probing by 
delving into the subtler aspects of individual 
or group learning. The acquisition and 
dissemination of data from these assess- 
ments may begin to bridge the gap between 
educational theory and practice. Assess- 
ment data can also help researchers and 
educators explore varying hypotheses about 
effective teaching and learning outcomes. 

Using Information Technology to 
Assess Student Learning 

While information technology is in- 
creasingly used to improve undergraduate 
teaching and learning, its use in assessment 

6 r\ 
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is lagging. Helgeson and Kumar (1993) 
reported that the most common use of com- 
puter technology in assessment was to ad- 
minister multiple-choice tests. More re- 
cently, of the 42 university presentations at 
an undergraduate program directors meeting 
of the Howard Hughes Medical Institute 
(HHMI), only one mentioned using infor- 
mation technology in student assessment, 
and that involved delivering a series of mul- 
tiple-choice items (HHMI, 1996). This 
lack of use in assessment reflects the imma- 
turity of information technology’s integra- 
tion into the wider educational arena. It 
also underscores the fact that evaluation 
strategies must parallel new classroom prac- 
tice and learning goals (Linn, Baker, and 
Dunbar, 1991) and not be neglected as in- 
formation technology enables a paradig- 
matic shift in postsecondary teaching and 
learning. 

Information technology offers cost and 
scale benefits, decreased reporting time, and 
increased validity in assessment of 
postsecondary student learning. Informa- 
tion technology also has the potential to: 

• provide continual rather than dis- 
crete measures of student learning; 

• guide assessments of student learn- 
ing in complex, real-world activities; 

• increase the quality of assessment 
data while reducing costs; and 

• provide measures of learning and 
skills among students with special needs. 



Formative Assessment Using 
Classroom Communication 
Systems 

Electronic classrooms can now make 
formative assessment feasible even in large 
lecture courses where the instructor may not 
know students by name. Classrooms 
equipped with communications systems 
such as Classtalk 17 (see Box A-9) allow 
instructors to assess student understanding 
at crucial points during the class period, 
thus encouraging learning that is consistent 
with the research presented in this section. 
Traditional lectures are best suited for those 
students who make sense of ideas while 
listening and taking notes. Relying on a 
single instructional strategy that helps only 
a portion of students to learn creates inequi- 
ties for all students, particularly those who 
assimilate materials better by discussing it, 
writing about it, and using it to solve prob- 
lems (Dufresne, Gerace, Leonard, Mestre, 
and Werk, 1996). 

When students discuss their answers in 
groups before responding to a query, their 
collective understanding is an example of 
distributed cognition. By using information 
technology to promote active engagement, 
e.g., by allowing students to run a simula- 
tion or visualize some phenomenon prior to 
engaging in discussions, SME&T faculty 
members improve the alignment between 
instruction and learning research. 



17 Available: http://www.bedu.com/. [7/27/01.] 
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BOX A-9 Classtalk 
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• c . Classtalk consists of a main com-fp| 

puter, a set of desktop or palmtop com- 
puters or calculators, and a network 
that connects the two components. 

• • The instructor prepares questions be- 

forehand, integrates them into the lec- 

,ture, and poses a question either ver- 

%'r. ' ;v .V*?/ • ■ • ' ♦ ' - > v 

bally or via the network. Most often, 

k®:qualitative 4 imultiple-ehoice questions 
“ are used to check student understand- 

• > • - t vyir-', : ,V “f •.; 

inq ?df pertinent curreht*concepts. Stu- 
dents^may be asked To discussSthe 
question among themselves before re- 
t Aspondinq via their< computers or calcu- 

lators. The instructor's computer keeps 

* 

track of the responses and can display 
them to the whole class, if desired, in 

WA'S : = 

the form of a histoqram or can show 
the instructor individual' responses in 
the seating chart. At noitime does the 
i whole class see an individual student’s 
response, which increases the likeli- 
iThOod oSparticipation and gives the in- 
structor a more realistic assessment of 
r T-'hovy.-well- Uae -.class • as^|^ple;uncler- 



■ 
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stands a particular concept. The com- 
puter provides another advantage not 
found in similar nontechnological ap- : 
l|: proaches. Mazur (1997) was among 
the first to report on Classtalk, and he 



ceptual understanding, he has a much 
eas,er t,me convincing h,s colleagues 
of the validity of the data 




BHiRCE: Mazur, 1997. 
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Like a fine arts portfolio representing an 
artist’s range and quality of work, a portfolio 
used for assessment includes samples of work 
that represent a student’s best efforts. The 
portfolio could include assignments, project 
reports, reports on significant events, results 
from experiments, and reflective notes in 
journal format. In short, a portfolio is a 
collection of evidence that learning has 
taken place. Because portfolio assessment is 
designed for integration with instruction, it 
is becoming one of the most appealing forms 
of assessment, and is one area where in- 
creased use of technology in the classroom 
can translate directly into modified assess- 
ment activities. 

Some of the benefits of portfolio assess- 
ment include: 

1 . By showing what they can do 
through their portfolios, students demon- 
strate skills and competencies for teachers, 
parents, potential employers, and even 
policy makers. 

2. Portfolios provide useful information 
to evaluate comprehensively the quality of 
education and the quality of student 
achievement. 

3. Because portfolios offer students a 
variety of ways to demonstrate what they 
know and can do, students are steered to- 
wards becoming reflective learners respon- 
sible for their own growth, thus encouraging 
and supporting multiple learning styles. 

4. Portfolios offer educators multiple 
avenues to understand what and how their 
students are learning, thus contributing to 
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faculty efforts to design appropriate instruc- 
tion that can improve student achievement. 

Digital portfolios are an example of an 
incremental enhancement to an existing 
assessment based on information technol- 
ogy. The types of materials that might be 
included in a digital portfolio range from 
student-teacher e-mails to more complex 
materials such as graphics or presentations 
created with multimedia software tools. 

(See, for example, Box A- 10.) Opportuni- 
ties for improving assessment using digital 
tools depend primarily on the creativity and 
ingenuity of faculty and the institutional 
support and rewards that administrators put 
in place to encourage using these kinds of 
assessment tools. 

Portfolios can require substantial effort 
on the part of students to produce and on 
the part of teachers to evaluate. Because of 
the new spectrum of skills displayed by stu- 
dents in portfolios, new rubrics to evaluate 
them also are needed (Gearhart, Novak, 
and Herman, 1996). 



Assessing Problem Solving with 
Information Technology 

Computer-based tools can be cost-effec- 
tive in assessing students’ problem-solving 
skills. For example, the Interactive Multi- 
media Exercises (IMMEX) Project at the 
University of California at Los Angeles 
School of Medicine, 18 which was created to 



18 Available: http://www.coled.umn.edu/edutech/ 
immex/. [8/8/01]. 



BOX A-l 0 Digital Portfolio 
Assessment at Valley City 
State University 



Valley City State University (Valley 
City, North Dakota) obtained a $1.7 
million Title III grant to provide instruc- 
tional technology training and equip- 
ment for its faculty and students. By 
the year 2000, all students who gradu- 
ate will have prepared a digital portfo- 
lio that .documents their best college 
work. This is possible in part because 
all full-time students are issued a laptop 
computer for their personal use. Stu- 
dents use CD-ROM-based portfolios to 
demonstrate their competence levels in 
a set of eight abilities that focus on the 
students' capacity to use the knowl- 

~ edge gained in the classroom: Ti . 

-• ' ’* 0 $''. ,;V Tfe.' .'; . 

Communications 

Problem solving and decision making 
Collaboration * 

Technology employment 
Effective citizenship 
Aesthetic responsiveness 
Global perspectives^ " ; 

r Welln esS >' f * 









The graphic capabilities of the CD-ROM 
technology enhance|students' capac- 

• . . •. 4JpC- -v: s' 'j&t T'Vv ; 

ity to record progress and experiences. 



assess medical students’ ability to diagnose a 
wide variety of illnesses, has evolved into a 
problem-solving platform for students from 
elementary school onwards. IMMEX is a 
software package with three components: 
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1. An authoring tool that instructors 
use to design customized problems in their 
particular discipline; 

2. A user interface for students, who 
are given a problem and a menu of choices 
for accessing information needed to solve 
the problem; and 

3. An analysis tool that records each 
student’s problem- solving actions in se- 
quence. 

As an assessment instrument, the most 
important component of 1MMEX is the 
analysis tool, which contains embedded 
artificial neural networks that are “trained” 
to recognize problem-solving patterns. 
Stevens, Lopo, and Wang (1996) used this 
component of 1MMEX to assess the ability 
of hundreds of UCLA medical students to 
correctly diagnose infectious diseases. As 
the first step in this process, infectious dis- 
ease specialists completed six computer- 
based clinical diagnosis simulations. Next, 
the 1MMEX analysis tools analyzed the diag- 
nostic strategies used by these experts, and 
identified common features of their solu- 
tions to a problem. Then, these expert- 
trained neural networks were used to assess 
the diagnostic abilities of medical students. 
Only 17 percent of the medical students 
were classified as demonstrating the same 
strategies as the experts. 

This 1MMEX assessment has informed 
medical school faculty about areas for im- 
provement in their efforts to train students 
to become capable diagnosticians. Informa- 
tion technology has become central to the 
medical school’s ability to perform learning 



assessments of individual students that oth- 
erwise would be time- and cost-prohibitive. 



CONCLUSION 

The examples of curricular innovations 
based on information technology described 
in this section are by no means exhaustive. 
Nevertheless, they illustrate the range of 
ways that individuals, departments, institu- 
tions, and consortia have used information 
technology to give their students new, more 
diverse, and more authentic learning experi- 
ences. These innovations also are consis- 
tent with recommendations made in 
SME&.T reform documents that encourage 
the following pedagogical and learning 
strategies: 

• visualization 

• simulation 

• real-world problem solving 

• collaboration 

• inquiry 

• design. 

The projects also are consistent with what 
research has shown about how students 
learn. Finally, many of these information 
technology-based innovations include ap- 
propriate ongoing assessment and evalua- 
tion. They are the result of careful, deliber- 
ate planning and adequate support, features 
that characterize successful, sustainable 
reform efforts. 

The next section provides an overview 
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of current research on teaching and learn- 
ing, using examples from this section to 
illustrate how educational research findings 
can be applied in the classroom. 

LEARNING USING 

INFORMATION 

TECHNOLOGY 



INTRODUCTION 

The committee’s fundamental premise is 
that information technology (IT) can be- 
come a tool to enhance undergraduate sci- 
ence, mathematics, engineering, and tech- 
nology (SME&T) learning for all students. 
To achieve this objective, pedagogy that 
includes information technology should 
draw on models of cognition that recognize 
the learning needs of today’s student popu- 
lation. 

This section of the report describes the 
committee’s survey of recent educational 
research and theories. It also examines 
possibilities for using IT to apply these theo- 
ries in undergraduate SME&T classrooms. 
Based on the examination discussed below, 
the committee found that IT can be an 
important vehicle to engage students and 
help them develop a deeper understanding 
of complex scientific and technical con- 
cepts. 



EDUCATIONAL RESEARCH 
AND CURRICULUM REFORM 

Traditional postsecondary instruction 
has relied primarily on oral presentation of 
fundamental science and engineering con- 
cepts, sometimes accompanied by a labora- 
tory component for students to verify the 
basic principles presented during lectures. 
These mostly passive learning environments 
can help some postsecondary students learn 
effectively. However, the growing need for 
a more scientifically literate (National Re- 
search Council [NRC], 1999b) and more 
technologically fluent (NRC, 1999a) citi- 
zenry, combined with the increasingly 
multicultural backgrounds of the 
postsecondary student population, call those 
traditional methods into question. Current 
students, including those interested in be- 
coming future scientists, mathematicians, 
and engineers, are reporting their dissatis- 
faction with teaching methods and course 
structure as well as obvious mismatches 
between their goals and those of the instruc- 
tor (e.g., Tobias, 1990; Seymour and 
Hewitt, 1997). 

To face this issue directly, and fueled in 
part by advances in neuroscience and learn- 
ing theory, educational researchers have 
developed a deeper understanding of how 
people learn at different ages and have 
documented the importance of active en- 
gagement in the learning process (Millar et 
al., 1996; Springer et al., 1997a; NRC, 
1999b). Fischer (1996) summarizes five 
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fundamental assumptions about learning (p. 

125): 

• Learning is a process of knowledge 
construction, not of knowledge recording or 
absorption. 

• Learning is knowledge -dependent; 
people use their existing knowledge to con- 
struct new knowledge. 

• Learning is highly tuned to the situa- 
tion in which it takes place. 

• Learning needs to account for dis- 
tributed cognition, requiring knowledge in 
the head of the individual learner to be 
combined with knowledge of the world. 

• Learning is affected as much by mo- 
tivational issues as by cognitive issues. 

The implications of these educational 
research findings for postsecondary SME&T 
educators are becoming more widely under- 
stood and, consequently, are fueling reform 
initiatives. Many of the curriculum initia- 
tives stemming from the reform movement, 
including those that make extensive use of 
information technology, have drawn on 
research in cognitive and social psychology, 
science and mathematics education, and 
ethnography and anthropology. This litera- 
ture has stressed that students come into 
courses with definite beliefs and preconcep- 
tions about how the world works and that 
their conceptual development may involve 
a process of resistance, compromise, and 
then assimilation of new or competing 
knowledge. 

Instructional activities also build on 
existing knowledge about the: 



• context of learning (situated cogni- 
tion); 

• acquisition of knowledge across a 
group of individuals (distributed cognition); 
and 

• influences of gender, ethnic and 
cultural influences; learning style; and moti- 
vation on student learning. 

There have been significant advances in 
understanding the process of knowledge 
acquisition in the last two decades. Learn- 
ing improves when instruction is 
contextualized (Brown, Collins, and 
Duguid, 1989); when it is driven by student 
interests and prior experiences (Shymansky 
et al., 1997); and when a variety of peda- 
gogical methods are used to reach students 
with a diversity of learning styles (Felder, 
1993). 

As faculty members shift their pedagogy 
to the kind of instruction that is grounded 
in cognitive theories of learning, informa- 
tion technology can be a powerful instru- 
ment to help achieve these ends. As infor- 
mation technology plays a larger role in and 
out of the classroom, it will provide both 
structured and unstructured learning experi- 
ences. Instructors can incorporate more 
student- student collaboration and problem- 
solving activities into their courses to meet 
the educational needs of a diverse student 
population. Educators also can use informa- 
tion technology to assess student learning 
and use this information to improve teach- 
ing and learning. 

12 
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LEARNING THEORIES AND 
EDUCATIONAL TOOLS 

A variety of learning theories have been 
proposed based upon a large body of educa- 
tional research. However, three theories of 
learning are particularly relevant to learning 
in the natural sciences: constructivism, 
situated cognition, and distributed cogni- 
tion. 

Constructivism is a model of learning 
that asserts that knowledge is not passively 
received but is actively created inside the 
mind of every learner. 

Situated cognition contends that knowl- 
edge and skills are best acquired in contexts 
from daily life (Brown et al., 1989). For 
example, Wilson, Teslow, and Osman- 
Jouchoux (1995) argue that because knowl- 
edge and learning are dependent on con- 
text, the content of knowledge within a 
population of learners is uneven. 

The central theory of distributed cogni- 
tion contends that learning is distributed in 
an environment and among learners and is 
not the product of isolated individual cogni- 
tive activity (Jonassen, Campbell, and 
Davidson, 1994). Therefore, learners con- 
struct understanding by using a variety of 
resources, including other people. 

Other factors not accounted for in these 
theories also can affect the way individuals 
learn. Gender, for example, can affect learn- 
ing by shaping preferences for information 
presentation and processing. The research 
literature contains numerous well-docu- 
mented studies of differences in learning 



styles of male and female students. Simi- 
larly, individual differences in motivation to 
learn and achieve can profoundly impact 
student learning and need to be considered 
in developing technology-enhanced cur- 
ricula. These factors, as well as the three 
educational theories of constructivism, situ- 
ated cognition, and distributed cognition, 
are discussed below. 

Constructivism 

Theoretical Background 

According to the theory of 
constructivism, learning is a dynamic pro- 
cess in which an individual actively inte- 
grates new information with existing knowl- 
edge. This idea has its roots in Jean Piaget’s 
groundbreaking research on cognitive devel- 
opment in children (Herron, 1978). Piaget 
was the first to propose that learning has 
two essential components. First, learners 
must take in information and compare it 
with the information already stored in their 
brain, a process known as assimilation. Sec- 
ond, when the new information is foreign or 
when it is at odds with what is already 
known, the learner must create a new men- 
tal representation in a process called accom- 
modation. Cognitive growth occurs when 
new mental representations are created or 
modified to fit the demands of reality. Ex- 
tensions of Piaget’s work have provided 
more precise definitions of these mental 
representations, or schema. Anderson 
(1980) describes schemas as “large, complex 
units of knowledge that organize much of 



what we know about general categories of 
objects, classes of events, and types of 
people.” 

According to proponents of 
constructivism, meaningful learning requires 
deliberate, personal restructuring of one’s 
conceptual framework. Cognitive scientists 
have elaborated on Piaget’s theories by de- 
veloping several learning models. One 
model, proposed by Shymansky et al. 

(1997), proposes that learning consists of six 
phases (Box A-ll). Three essential phases 
precede what Piaget referred to as the as- 
similation phase. During conceptual equi- 
librium, learners are comfortable with their 
present understanding. They then have an 
experience with a new body of information 
that challenges their conceptual under- 
standing and leads to disequilibrium. As- 
similation then occurs when learners adopt 
the new information to reestablish equilib- 
rium. Development of new schema in ac- 
commodation is then followed by re-equili- 
bration, a stage that persists until the next 



BOX A- 11 The Six Phases ' * 
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challenge to the learners’ existing knowl- 
edge structures. 

Through these six phases, learners must 
integrate new information with prior knowl- 
edge to build more elaborate knowledge 
structures or schema. The role of the 
learner’s prior experiences in developing 
conceptual understanding has been well 
documented (Bodner, 1991). Furthermore, 
research indicates that highly persistent 
misconceptions can arise from erroneous 
original learning, which forms a portion of 
the foundation of subsequent information 
that the student perceives as related to the 
existing knowledge (Zoller, 1996). Learners 
often retain erroneous models and explana- 
tions because they seem more reasonable 
and more useful to the learner (Mayer, 
1987). In many cases, if not challenged, 
these beliefs can persist, potentially hinder- 
ing further learning (McDermott, 1991; 
Nakhleh, 1992). 

Once learners discard their misconcep- 
tions, they must then integrate their new 
knowledge with what Wittrock (1974) calls 
“generative processing.” Although faculty 
members often “tell” students things, such 
as factual information in a large lecture 
setting, the students must become person- 
ally engaged with the material and perform 
mental operations on this new knowledge 
before the knowledge becomes their own. 
Robust learning best occurs when the stu- 
dent can examine and test the information 
received (Bellamy and McNeill, 1994; 
Wang, 1996). SME&T faculty members 
can select information technology-based 
instructional strategies to help their stu- 



dents interpret new information, reason 
from what is known, and solve complex 
problems. 

Constructivism and Information 
Technology-Based Teaching and 
Learning 

Models for instructional design, teaching, 
and learning need to reflect the ways scien- 
tists, mathematicians, and engineers create 
new knowledge. Likewise, these models 
must be grounded in current knowledge 
about how students learn as well as the con- 
text in which that learning takes place. In 
the decades since Atkin and Karplus (1962) 
first proposed the concept of a learning 
cycle, a variety of instructional models have 
become available to link teaching decisions 
with learning research (National Center for 
Improving Science Education, 1991). 

Bybee (1997) proposed one such model, 
known as the “5 Es” model, which contains 
the following elements (pp. 178-179): 

1. Engage students in a learning task. 

2. Enable students to explore their 
ideas. 

3. Guide students to devise an Expla- 
nation for the observations and results of 
their exploration. 

4. Elaborate on students’ experiences 
by extending or applying the learning to 
new situations. 

5. Evaluate students’ explanations and 
understanding. 

The 5 Es model can guide faculty in 



selecting instructional strategies that help 
learners overcome their misconceptions. 
When the learning task includes discrepant 
events that yield unexpected results, learn- 
ers will try to understand their observations. 
Their exploration will naturally lead to 
proposals for explanations, some of which 
are now no longer plausible or feasible in 
light of the unexpected results. By letting 
students apply what they have learned to 
new situations, the instructor can help the 
students reinforce new conceptual under- 
standings and permanently discard old, in- 
correct ones. 

Many of the examples of undergraduate 
SME&.T reform initiatives described in the 
previous section are grounded in educa- 
tional research about student learning. For 
example, the use of the computer as a data- 
collection device is consistent with 
constructivist learning theory. Three pro- 
grams (Studio Physics, Workshop Physics, 
and the Center for Science and Mathemat- 
ics Teaching) extensively use Microcom- 
puter-Based Laboratory (MBL) sensors and 
software. Students in these programs use 
the MBL tools to measure and graph such 
physical quantities as position, velocity, 
acceleration, force, temperature, light inten- 
sity, pH, pressure, sound pressure, radiation, 
current, and voltage. They use the results to 
construct their own understanding of physi- 
cal phenomena in a process that mirrors the 
5 Es instructional model. Students receive a 
scenario, discuss the scenario with other 
group members, suggest predictions about 
the outcome, and then use real-time mea- 
surement tools to collect preliminary data. 
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Unexpected results occur frequently, which 
lead the group to more rigorous experimen- 
tal procedures as members struggle to make 
sense of an outcome that seems 
counterintuitive. Extensive investigation of 
this type of student learning demonstrates 
that instruction employing information 
technology can provide students with a 
conceptual understanding that is superior to 
other modes of instruction (Laws, 1991). 

Construction of dynamic computer 
models is another means of implementing 
the 5 Es instructional paradigm. Such mod- 
els encourage students to analyze, synthe- 
size, reason, and explain complex systems 
(Spitulnik and Krajcik, 1998). Using infor- 
mation technology this way also introduces 
students to scientists’ uses of models, as 
advocated by national science education 
reform efforts (e.g., American Association 
for the Advancement of Science, 1993). 
Casti (1997) explains why scientists use 
models: 

• to predict a system’s behavior in the 
future, based on the system’s properties and 
current behavior (a predictive model); 

• to provide a framework for under- 
standing past observations as part of an 
overall process (an explanatory model); and 

• to offer a picture of the real world 
with additional features built in, to let users 
bend or shape that reality to their own lik- 
ing (a prescriptive model). 

Each type of model enables learners to 
understand the connections between real- 
world concepts by constructing an environ- 
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ment to formulate and test a phenomenon 
(Spitulnik and Krajcik, 1998). 

Situated Cognition 

Theoretical Background 

According to constructivism, learners 
use prior experiences and existing schema to 
understand new information and create new 
knowledge. The theory of situated cogni- 
tion expands that premise by examining the 
context and culture in which knowledge is 
constructed. A central tenet of situated 
cognition is that learning is affected by the 
environment (Brown et al., 1989; Carr, 
Jonassen, and Litzinger, 1998; Jonassen et 
al., 1994). For example, some Brazilian 
school-age children assist their street- 
vendor parents by performing mathematical 
calculations to determine the total of a 
customer’s purchase, despite their limited 
formal education (Carraher, Carraher, and 
Schliemann, 1985). The context of street 
vending contributes to developing math- 
ematical skills. 

Researchers once viewed the process of 
acquiring knowledge and the context in 
which it is applied as two separate entities. 
Proponents of situated cognition challenge 
this premise and assert that the environ- 
ment should reflect how the knowledge is 
used (Brown et al., 1989), so that acquiring 
and applying knowledge becomes a cohe- 
sive, integrated process (Brown et al., 1989; 
Jonassen et al., 1994). Information acquired 
apart from its context is unusable or “inert” 
(Griffin and Griffin, 1996; Jonassen et al., 




1994; Young, 1993). In the aforementioned 
example, the marketplace gives meaning to 
the mathematical calculations performed by 
Brazilian children. 

Proponents of situated cognition often 
use the term “cognitive apprenticeships” to 
describe experiences in which learners en- 
gage in “authentic practices” through activi- 
ties or social interactions. The term is remi- 
niscent of a craft apprentice who learns 
from a more experienced tradesperson 
(Brown etal., 1989; Wilson, 1993). Ulti- 
mately, the goal of cognitive apprentice- 
ships is to promote learning in the context 
of an activity, culture, or tool — to allow 
learners to become “enculturated in a com- 
munity of practice” by observing and experi- 
encing the values and practices of a profes- 
sional working community (Carr et al., 

1998, p. 6). The concept of cognitive ap- 
prenticeships supports the idea that indi- 
viduals must experience the culture and 
context for meaningful learning to occur 
(Wilson, 1993), which in fact happens in 
most graduate research in SME&T disci- 
plines. Unlike traditional lecture instruc- 
tion, cognitive apprenticeships encourage 
learners to develop knowledge through the 
modeling, coaching, and guidance of men- 
tors (Wilson, 1993). Moreover, cognitive 
apprenticeships enable learners to collabo- 
rate with practitioners and gain access to 
the wisdom of mentors. In joining a com- 
munity of practice, learners also develop 
their own identity as members of the com- 
munity (Lave and Wenger, 1991). 

Not all educators fully subscribe to situ- 
ated cognition as a theory of learning. For 



instance, Tripp (1993) acknowledges that 
teaching concepts without context is an 
exercise in futility, but he questions the 
emphasis placed on immersing students in a 
“community of practice” without the sup- 
port of classroom experience. Without a 
balance of authentic and classroom experi- 
ences, Tripp argues, students may gain only 
partially developed skills for a setting or 
activity. Anderson, Reder, and Simon, 
(1996) advocate balance when applying the 
principles of situated cognition, asserting 
that learning is both context dependent and 
context independent. They maintain that, 
while some skills develop best within the 
appropriate social context, other skills may 
be developed better via abstract instruction 
in classroom settings. Overall, researchers 
who are cautious about situated learning are 
not opponents of the theory per se but are 
interested in a feasible balance between 
practical and instructional experiences. 

Situated Cognition and Information 
Technology-Based SME&T 
Learning 

The examples of successful information 
technology-based innovations in 
postsecondary SME&T education described 
in the previous section of this appendix are 
consistent with research into effective, effi- 
cient learning, especially for students who 
historically have been underrepresented in 
the SME&T disciplines. Many of the ex- 
amples cited are grounded in realistic, prac- 
tical problems in science and engineering. 
These kinds of problems not only can help 



students integrate new information into 
their existing knowledge base but also can 
enhance their motivation to learn by pro- 
viding a relevant context. The committee’s 
examples of information technology-based 
courses include several instances of students 
working with real data on real problems of 
interest to professional scientists. 

Distributed Cognition 

Theoretical Background 

Psychology has identified cognition as 
occurring “inside the head” of individuals, 
with only secondary influences from social, 
cultural, and historical factors. In effect, 
thought processing takes place in isolation 
from the context of the thought. According 
to Salomon (1993), such a perspective is 
appropriate for examining information- 
processing models but does not necessarily 
represent daily situations involving collabo- 
ration with others and the use of available 
tools and resources. 

In contrast, the theory of distributed 
cognition holds that cognition is distributed 
across people, environments, situations, and 
objects, all of which are referred to as arti- 
facts. Artifacts serve as vehicles that repre- 
sent or act on information (Hunt, 1992; 

Pea, 1993). In essence, artifacts alter how a 
person performs a task and may compel the 
individual to employ entirely different cog- 
nitive skills to complete the task (Norman, 
1991). 

While many researchers use the term 
“distributed cognition,” Pea refers to the 



same concept as “distributed intelligence” to 
make the distinction that “people, not de- 
signed objects, ‘do’ cognition” (Pea, 1993, p. 
50). For the discussion that follows, “dis- 
tributed cognition” and “distributed intelli- 
gence” are used interchangeably. 

In his study of distributed cognition 
among airline crew members, Norman 
(1992) identified numerous instances where 
crew members used common objects as arti- 
facts to communicate information. This 
study was motivated by a tragic circum- 
stance. A plane crashed — reportedly, be- 
cause the captain had a heart attack that 
went undetected by the first officer, who 
dismissed the captain’s unresponsive behav- 
ior as typical and, therefore, not alarming. 
This example illustrates several principles of 
distributed cognition, including the danger 
of having all the knowledge reside in one 
individual (Norman, 1992). As with so 
many activities, a commercial plane cannot 
be flown by a single individual but requires 
collaboration among all crew members. 
Distributed cognition also discourages cog- 
nitive overload, in which individuals either 
fail to notice critical pieces of information 
or concentrate on a single explanation to 
the exclusion of other possibilities. 

Distributed Cognition and SME&T 
Learning 

Group learning, in which students draw 
on and learn from the experiences of others, 
is one manifestation of distributed cogni- 
tion. Collaborative learning and coopera- 
tive learning are two forms of group learn- 
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ing. Although similar, the concepts are not 
synonymous. Collaborative learning is 
. .an umbrella term for a variety of educa- 
tional approaches involving joint intellec- 
tual effort by students, or by students and 
teachers together” (Goodsell et al., 1990). 
Cooperative learning (Box A-12), a form of 
collaborative learning, is an instructional 
technique that allows students to work in 
groups to achieve a common goal, to which 
they each contribute in individually ac- 
countable ways (Slavin, 1995). Learning 
increases when students learn interactively 
in groups (Brufee, 1993). For example, in a 
meta-analysis of published research on un- 
dergraduate SME&T education, Springer et 
al. (1997b) found that small-group learning 
results in greater academic achievement, 
better attitudes toward learning, and in- 
creased persistence in SME&T courses. 
Johnson, Johnson, and Smith (1998) con- 
ducted a similar meta-analysis and came to 
the same conclusion — namely, that coop- 
erative learning promotes higher individual 
achievement than either competitive or 
individualistic approaches to learning. 

These findings seem to hold regardless of 
discipline or class size. Group learning in 
biology (Watson and Marshall, 1995), 
chemistry (Wright, 1996), earth science 
(Macdonald and Korinek, 1995), and phys- 
ics (Hake, 1998) all resulted in greater stu- 
dent achievement than did courses where 
students worked independently. 

The committee reviewed several ex- 
amples of curricula that incorporate collabo- 
rative learning techniques. The Workshop 



BOX A-l 2 Key Elements of 
Cooperative Learning 

■ Students understand that their 
success depends on the success of oth- 
ers in their group. 

■ Activities and assessments are 

structured to allow for individual ac- 
countability, *• .. «. ... 

• Students are expected to promote * 
one another's success. •■•••*»• .; 

■ Faculty members teach students 
the social skills needed for cooperative 
endeavors. *• -y * »r ; 

• .Students areyencouraged to fe~ 1 
fleet on the group's learning process. *• ? 

A r> - :5S ■ •••;« ^ 

SOURCE: Johnson et al., 1 998. * 



Physics curriculum 19 uses cooperative learn- 
ing strategies for guided inquiry activities 
and student projects. Likewise, students 
using BioQuest curriculum materials 20 work 
in groups. The University of California at 
Los Angeles’ Science Challenge 21 offers 
lower-division students hands-on experi- 
ence using real scientific problems, and 
provides support structures that foster team- 
work rather than pitting students against 



19 Available: http://physics.dickinson.edu/ 
PhysicsPages/Workshop_Physics/Workshop_Physics. 
Home.htm. [7/27/01]. 

“Available: http://bioquest.org. [7/27/01.] 

2 'Available: http://www.nslc.ucla.edu/SciChal. 
html. [7/27/01]. 
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their peers. These and other information 
technology-based curriculum innovations 
use a distributed-cognition learning model. 
Group members are responsible for gather- 
ing data on a facet of a problem, sharing 
that data and their knowledge with other 
group members, and integrating that knowl- 
edge into the overall solution to the prob- 
lem. 

The examples identified also enable 
students to connect to rich resources on a 
network and communicate with their peers 
outside the classroom, consistent with the 
precepts of distributed cognition and, to 
some degree, situated cognition. Class ac- 
tivities that incorporate distributed intelli- 
gence encourage students to assume greater 
ownership and participation in learning. 
Students actually become “inventors of 
distributed-intelligence-as-tool, rather than 
receivers of intelligence-as-substance” (Pea, 
1993, p. 82). Information technology af- 
fords students the resources to tackle their 
assignments more fully by accessing data- 
bases, the Internet, and other materials. 

And the mark of a successful student is gen- 
erally the one who accesses a variety of di- 
verse resources and then uses the informa- 
tion inventively to solve problems and 
complete tasks (Fischer, 1996; Pea, 1993). 

In the BioQuest instructional modules, 
complex simulations of real phenomena are 
used to generate data for student problem 
solving. The processes students use to solve 
a problem, including the resources they 
access, also provide the instructor with a 
great deal of insight about student under- 
standing of the problem. In Project Interac- 



tive Multimedia Exercises, SME&T educa- 
tors can use information technology to track 
student analytical skills by documenting 
each step in problem solving, which is virtu- 
ally impossible to do with any other 
method. Harley (1993) points out that in 
situated learning settings such as this, the 
instructor s role becomes supportive rather 
than directive. Such a role can be equally 
demanding, since the instructor must deter- 
mine which problems to explore, provide 
the necessary background learning or “scaf- 
folding,” and assess and reassess the situated 
learning that occurs (Young, 1993). 

Distance learning courses also illustrate 
the application of distributed cognition. For 
example, Professor Kawagley’s “Native 
Ways of Knowing” at the University of 
Alaska enable students who are separated 
both physically and culturally to engage in 
cooperative learning (see Box A-13). The 
emphasis in the Alaska example is not on 
the thought process and work of individuals 
per se but more on how group members ex- 
change thoughts and ideas to complete the 
assignment. This project exemplifies the 
essence of distributed cognition, in which 
thought processing occurs between group 
members and by means of a diverse set of 
resources. 

Gender Differences in the Learning 
Process 

Theoretical Background 

An article in The Washington Post, head- 
lined “Gender Gap in Fairfax Computer 
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|BG)X^- 13 ^Native Ways of 
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Xk the University. of|Vaska, Professor 
Angayuqaq Kawagleylqffers a cross- 
listed Native Studies/Education course 
that addresses the ways that Native 

Alaskan cultures use empirical knowl- v 

j* -•**- v3®* •naffly : JM' t-' r- 

edge to derive abstract knowledqe * 

about the natural world and link this 

- ^ : — * 



with modern scientific knowledge. 1 




'^Pus ethnic groups from communities^ 
separated by hundreds of miles partic- 
ipate in the course. Class "meetings-^ - 
are broadcast live over television 



throughout rural Alaska, creating an 
unexpectedly large unenrolled audi- 
ence. , rWCI-i'?? * 

The technology allowed students to 
remain \n their physical and cultural 
context during the course. Instead of 
the usual difficulties that diverse learn- 
ers face in relating course content to 
their personal circumstances, contex- 
tual interpretation was explicitly dis- 
cussed. Discussion of context added a 
critical dimension to learning about sci- 

" ■ -r- ' , ■ 

ence. Together, students realized that 

single perspectives or explanations of 
. . ■■ , 
jral phenomena are incomplete; 

:ceptmg all explanations simulta- 

eously is incoherent. Pressed to con- 

der the logic internal to their own 

jlture, each student also could press 

ror the same coherence from other 
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Classes,” reported on the disproportionate 
number of male students who take high 
school computer classes in the Fairfax 
County, Virginia, school system when com- 
pared with the number of female students 
enrolled. 22 The 3-to-l ratio of males to 
females in these classes is part of a national 
trend, according to a report by the Ameri- 
can Association of University Women 
(AAUW, 1998). AAUW convened the 
AAUW Educational Foundation Commis- 
sion to Examine Gender, T echnology, and 
Teacher Education in order to study the 
underlying causes of gender difference in 
computer usage. 23 Do males and females use 
computers differently? Do teachers contrib- 
ute to gender-related differences in students’ 
attitudes toward computers and learning 
with or about computers? What can be 
attributed to the fact that computers and 
user interfaces were mainly developed by 
males (NRC, 1997a)? Are males more pre- 
disposed than females to exhibit strength in 
visual-spatial skills, which computer-based 
learning emphasizes? Regardless of the 
eventual outcome, studying these differ- 
ences will contribute to understanding and 
possibly correcting any inequities in access 
to high-technology. 

Research to understand gender differ- 
ences in learning has increased during the 



22 Benning, V. 1998. Gender Gap in Fairfax 
Computer Classes: Report Says Boys Outnumber 
Girls 3 to 1; Some Minorities Also Underrepre- 
sented. Washington Post. July 14:B1, B5. 

23 Available: http://www.aauw.org/2000/. 
[7/27/01]. 
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last 20 years and has included studies of a 
variety of age groups. These studies have 
ranged from examinations of verbal abilities 
to studies of visual-spatial skills in females 
and males. The finding that males consis- 
tently score higher than females on science 
tests (Howe and Dowdy, 1989) suggests the 
need for further research on gender differ- 
ences. Hamilton (1995) suggests several 
explanations for different SME&T achieve- 
ment between males and females. Although 
environmental factors and personal experi- 
ences can influence a student’s ability to 
leam and apply information, biological and 
physiological factors also can affect that 
learning, although to what extent remains 
under debate (NRC, 1999b). 

Neuroscientists who study gender differ- 
ences in learning have focused their investi- 
gations on the anatomy and physiology of 
the brain, particularly the lateralization 
(specialization) of the brain’s two hemi- 
spheres to perform specific tasks. Language, 
for example, is thought to reside primarily in 
the left hemisphere, while visual-spatial 
processing occurs predominantly in the 
right hemisphere. However, some research- 
ers have hypothesized that females have 
better verbal abilities because language re- 
sides in both hemispheres of the brain in 
women but is primarily lateralized in the left 
hemisphere in men. By using magnetic 
resonance imaging to track brain activity 
during language tasks, Shaywitz et al. ( 1995) 
demonstrated that these tasks primarily 
involved the left brain hemisphere among 
males, while, among women, the same re- 
gion of both hemispheres was active. These 



results are consistent with the current hy- 
pothesis about hemispheric specialization, 
but Shaywitz et al. (1995) also acknowledge 
that other regions in the brain may contrib- 
ute to phonological and language tasks. If, 
in fact, language is bilateralized across both 
hemispheres of the brain in females, then 
the theory of “cognitive crowding” proposed 
by Levy (1976) becomes more plausible. 
Levy argued that visual-spatial cognition, 
which is typically attributed to the right 
hemisphere, must share neural space with 
language functioning in the right hemi- 
sphere in females. In comparison, language 
functioning for males is lateralized to the 
left hemisphere, allowing more neural space 
in the right hemisphere for functioning 
related to visual-spatial activities. 

Gender Differences and 
Information Technology-Based 
Learning 

Although there is an extensive body of 
research from many areas of science that 
attempts to identify underlying biological 
explanations for differences in cognition 
between males and females, many studies 
have simply documented these differences 
without attempting to attribute them to 
either physiological or social factors. These 
studies are useful because they help to iden- 
tify fertile areas for research on the underly- 
ing causes of gender differences in cognition 
and, equally important, because they often 
have immediate, practical applications. 

For example, Hall and Hickman (1997) 
conducted a study in which 27 undergradu- 
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ate psychology students ranging from 19 to 
45 years old evaluated five Web pages that 
described a neuron. Three of the pages 
consisted of text, one page consisted of a 
picture of the neuron, and one page con- 
sisted of both the text and the neuron. Tire 
background of three of the pages varied in 
complexity, which allowed investigators to 
investigate the effects of both the form of 
information and the context in which it was 
delivered. After reviewing the pages, stu- 
dents answered a questionnaire regarding 
their age, gender, experience with comput- 
ers, and experience with the World Wide 
Web. Results indicated that both sexes gave 
higher ratings to the Web pages that con- 
tained both the text and the picture of the 
neuron; however, results also revealed a 
significant difference between male and 
female ratings regarding the visual complex- 
ity of the Web pages. The men gave a 
higher rating to the more visually complex 
pages, while women rated those pages lower 
on the scale. While Hall and Hickman 
(1997) stated that females reported less 
experience with computers and the World 
Wide Web than did men, this factor did not 
seem to account for the significant differ- 
ence in the complexity ratings. 

Learning Styles 

Theoretical Background 

In an era of increasing student diversity, 
educators can enhance learning opportuni- 
ties for all students by being more aware of 
learning differences. It has been well estab- 



lished that individuals develop a preferred, 
consistent set of behaviors or approaches to 
learning (Litzinger and Os if, 1993). This 
set of behaviors, otherwise known as a 
learning style, can be described in terms of 
four layers (Curry, 1983), with different 
learning styles tending to concentrate on 
one of the layers: 

1. The personality layer describes an 
individual’s basic personality, often in a 
continuum from introvert to extrovert. 

2. The information processing layer refers 
to the way a learner prefers to take in and 
process information. 

3. The social interactions layer centers 
on how students behave and interact in the 
classroom (e.g., focused solely on learning or 
on grades). 

4. The instructional preference layer 
describes the mode in which learning occurs 
most easily (e.g., listening, reading, and 
direct experience). 

These learning style layers are not inde- 
pendent of each another, since the traits of 
one level will influence the next. Nonethe- 
less, cognitive psychologists have developed 
a number of models for each of the layers. 
For a comprehensive review of these mod- 
els, see Claxton and Murrell (1987), who 
focused on how these layers overlap; options 
for helping students determine and under- 
stand their own learning style; trends in 
gender-related differences in learning style; 
and the implications of this information for 
SME&.T faculty members. 

Two models of learning styles have be- 



come widely accepted, in part because of 
simple-to-use instruments that help deter- 
mine a student’s learning style. Kolb’s 
model of experiential learning (1984) de- 
scribes learning as a continuum from con- 
crete to abstract and from active to reflec- 
tive. The resulting 2x2 matrix yields four 
learning styles, as shown in Table A-2. 

Felder (1996) proposed another model 
of learning styles. In his model, five key 
questions are posed to probe a student’s 
learning preferences: 

• Does the learner prefer sensory infor- 
mation (sights, sounds, and physical sensa- 
tions) or intuitive information (memories, 
ideas, and insights)? 

• Does the learner perceive sensory 
information most effectively through visual 
or verbal modes? 

• Does the learner prefer to process 
information actively or reflectively? 

• How does the learner progress to- 
ward understanding, sequentially or glo- 
bally? 

• Is the learner more comfortable with 
information that is obtained inductively 
(principles inferred from facts and observa- 



TABLE A-2 Kolb's Learning Styles as a 
Continuum from Concrete to Abstract and from 
Active to Reflective 





Reflective 


Active 


Concrete 


"Why?" 


"What if?" 


Abstract 


"What?" 


"How?" 


SOURCE: 


Kolb, 1984. 
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tions) or deductively (consequences and 
applications inferred from principles)? 

Felder and Soloman 24 have developed a 
“Learning Styles Inventory” to help students 
answer the first four of Felder’s questions 
about their own learning. Like Kolb (1984) 
and Curry (1983), Felder (1993, 1996) sees 
these traits as a continuum and describes 
students according to their preference for 
one mode or another. He further argues 
that, although students may be uncomfort- 
able with instruction in their less-preferred 
modes of learning, faculty members should 
teach in multiple modes to accommodate 
the range of individual differences in a class 
(Felder, 1996). In the end, much of the 
responsibility for learning lies with the indi- 
vidual student. Knowledge about one’s own 
learning style is only useful if students know 
what to do with that information. After 
students complete the Learning Styles In- 
ventory, Felder provides them with a hand- 
out of tips for maximizing their learning in 
ways that are consistent with their own 
learning style. 25 

Learning Styles and SME&T 
Education 

Information technology-based instruc- 
tion has particular implications for the in- 



24 Available: http://www2.ncsu.edu/unity/lockers/ 
users/f/felder/public/ILSdir/ILS-a.htm. [7/27/01]. 

25 Available: http://www2.ncsu.edu/unity/lockers/ 
users/f/felder/public/ILSdir/styles.htm. [7/27/01]. 
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formation processing and instructional pref- 
erence layers in Curry’s model of learning 
styles (Curry, 1983). Information technol- 
ogy can help SME&.T faculty members ad- 
dress the learning needs of all of their stu- 
dents by offering students instruction in a 
more preferred mode. Active learners, for 
instance, like to discuss ideas with their 
classmates or solve problems in a group 
setting. Electronic communications tech- 
nologies make this possible even when stu- 
dents are in different locations. Information 
technology is especially helpful for visual 
learners, who gain more from pictures and 
diagrams than they do from the written and 
spoken words that fill most lecture classes. 
Sensory learners prefer facts and observa- 
tions to concepts and interpretations and 
can be frustrated by courses devoid of real- 
world context. 

Traditional lecture courses best serve 
students who are intuitive rather than sen- 
sory learners (Godleski, 1984). Faculty 
members who use information technology 
to provide science and mathematics instruc- 
tion are likely to enhance student retention 
in their courses by maintaining the interest 
and motivation of students with other learn- 
ing styles. The examples discussed in this 
document showcase how information tech- 
nology could present information in a vari- 
ety of ways. Computers offer unique infor- 
mation processing capabilities, such as 
multiple representations, interactive real- 
time assessment, and document revision, all 
of which can help individual students bet- 
ter construct their own understanding 
(Spitulnik and Krajcik, 1998). In addition, 



the use of multiple representations (dia- 
grams, drawings, graphs, animations, and 
video) more closely approximates the way 
that scientists understand and describe 
phenomena, and can help students under- 
stand scientific concepts (Kozma and 
Quellmalz, 1995). Multiple representations 
of information also allow faculty members 
greater flexibility and creativity in develop- 
ing effective pedagogical practices. 

Finally, the use of multiple representa- 
tions can make SME&.T courses more inclu- 
sive for all students, not just those who 
learn the same way as their instructors. 
Halpem (1992) emphasized that knowledge 
about learning styles can be used to facili- 
tate and improve curricula and instruction 
for both sexes. Material in SME&.T courses 
can be presented in multiple ways, and vari- 
ous aspects of problem solving can incorpo- 
rate both visual-spatial and verbal skills. 
Information technology-based initiatives 
that allow students to visualize complex 
phenomena, such as mathematical functions 
or molecular structures in three dimensions, 
can broaden the range of learning styles the 
instructor can accommodate. 

For example, visualization of math- 
ematical functions is central to the use of 
information technology in the calculus ini- 
tiatives described in the previous section. 
Project CLEA 26 is another program that 
makes use of visualization of complex phe- 
nomena to teach. It allows students to con- 



26 Contemporary Laboratory Experiences in As- 
tronomy. Available: http://www.gettysburg.edu/ 
project/physics/clea/CLEAhome.html. [7/27/01]. 



struct and manipulate images of stars and 
other astronomical bodies constructed from 
digital data. The Molecular Science Initia- 
tive at UCLA 27 uses molecular modeling 
hardware and software to provide students 
with images of phenomena that are other- 
wise impossible to visualize without infor- 
mation technology tools. Projects that fea- 
ture GIS usage or multimedia tools similarly 
enable instructors to reach students with a 
variety of preferred learning styles. 

The Role of Motivation in Learning 

Theoretical Background 

The role of science education is to help 
learners grow in all three educational do- 
mains: cognitive, psychomotor, and affec- 
tive. The development of problem-solving 
and reasoning skills and the acquisition of 
facts and concepts occur in the cognitive 
domain, while the development of physical 
and dexterity skills takes place in the psy- 
chomotor domain. The affective domain 
incorporates the learner s motivation, atti- 
tudes, and beliefs. 

The keys to success in education often 
are rooted in how a student feels about 
home, self, and school. These factors can 
confound efforts to transform postsecondary 
SME&T education because much of the 
affective domain is well established before 
students complete high school. Motivating 
students to learn science is perhaps the 



27 A variable: http://www.mosci.ucla.edu. [7/30/ 
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greatest challenge for SME&T educators. 
Unfortunately, current teaching practices 
often dampen student motivation and inter- 
est in science. The information technology- 
based initiatives highlighted in this docu- 
ment have been successful in part because 
they are sensitive to the affective dimen- 
sions of learning. The examples showcase a 
variety of instructional strategies and indi- 
cate how they accommodate a wide range of 
learning styles. 

Motivation affects the likelihood that 
students will persist in a course. If success is 
defined partly in terms of a student’s course 
completion, student motivation becomes an 
important consideration. 

McMillan and Forsyth (1991) define 
motivation as “purposeful engagement in 
classroom tasks and study to master con- 
cepts and skills.” Although this definition 
may need to be broadened to encompass 
non-campus based learning (vis-a-vis “class- 
room” tasks), it nonetheless suggests impor- 
tant research questions about the catalysts 
for engagement and student persistence 
mastering subject matter. 

As discussed above in the section on 
situated cognition, some motivation to learn 
derives from the link between the informa- 
tion being learned and its application. If 
the information seems irrelevant, students 
may not pay close attention to the details, 
and this can lead to misconceptions (Cogni- 
tion and Technology Group at Vanderbilt 
University, 1993). While there are several 
ways to define “relevance,” ultimately, it is 
in the mind of the learner. When courses 
and assignments incorporate the kinds of 
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experiences described in the previous sec- 
tion, increased motivation and “time on 
task” may help students focus not only on 
the “big picture” but also on those very de- 
tails that will help them build a solid foun- 
dation for further learning of SME&T. 

Student Motivation and 
Information Technology-Based 
Education 

Student interest and satisfaction with 
instructional materials also can enhance 
motivation. For example, multimedia ap- 
proaches can increase student motivation 
and interest (Fifield and Peifer, 1994; Pow- 
ers, 1998). Multimedia can be fun, and 
“having fun with learning may enhance 
motivation and achievement” (Druger, 
1997). Information Technology can also be 
used to tailor instruction to the needs and 
learning styles of individual students. In- 
struction that recognizes how much each 
student has already mastered and what sig- 
nificant questions remain can enhance mo- 
tivation and learning. 

For example, physicists at the U.S. Air 
Force Academy, Indiana University-Purdue 
University Indianapolis, and Davidson Col- 
lege have collaborated to develop Just in 
Time Teaching (Novak, Patterson, and 
Gavrin, 1999). In this approach, instructors 
develop web-based assignments, and stu- 
dents respond electronically. The instructor 
reads the student submissions “just in time” 
to adjust the lesson content and activities to 
suit the students’ needs. This creates a feed- 



back loop, in which student preparation 
outside of class fundamentally affects what 
happens during the classroom session. As a 
result, most students come to class already 
prepared and engaged with the material. 
And, the faculty member knows at the be- 
ginning of class exactly how much each 
student has learned and how to best spend 
the classroom time. Originally developed to 
support physics instruction, more than 60 
colleges and universities are now using this 
approach to provide instruction in a variety 
of academic disciplines (Novak, Patterson, 
and Gavrin, 2001). It appears that Just in 
Time Teaching motivates students to in- 
crease both the amount and quality of their 
interactions with other students and with 
faculty, as well as increasing their time on 
educational tasks. One extensive study 
found that these three factors — student- 
student interaction, student-faculty interac- 
tion, and time on task are critical to success 
in the undergraduate years (Astin, 1993). 

Motivation, attitudes, and beliefs are 
equally important for “traditional” and 
“nontraditional” postsecondary SME&tT 
learners and must be considered when de- 
veloping courses and curricula for nontradi- 
tional populations of students. Many re- 
searchers contend that the motivators for 
adult learners are quite different from those 
of other populations. For example, 
Cennamo and Dawley ( 1995) found that 
internal motivators such as self-esteem, 
quality of life, and increased job satisfaction 
are far more important to adult learners 
than external motivators such as grades. 
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CONCLUSION 



There is now considerable research on 
how students learn science and mathematics 
and how best to teach these disciplines (e.g., 
Gabel, 1990; NRC, 1999b). This research 
has contributed to the growing diversity of 
instructional strategies in postsecondary 
SME&T education (McNeal and D’Avanzo, 
1997; Glassick et ah, 1997). This trend must 
continue so that greatest potential learning 
gains from the use of information technol- 
ogy for teaching and learning SME&T can 
be achieved. 

The applications of information tech- 
nology that are described in this section are 
firmly grounded in the educational research 
literature. They are based on what is known 
about how students leam, both individually 
and in groups, and about how to establish 
inclusive learning opportunities that student 
find relevant to their personal experience 
and goals. These applications also share 
another important feature: considerable 
effort was expended on designing, evaluat- 
ing, and using the outcomes of those evalua- 
tions to improve each application. This 
process of continual improvement also has 
catalyzed further research. Each application 
has contributed to the growing body of lit- 
erature about educational assessment meth- 
ods. 
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Appendix B 

Workshop Agenda 



Workshop on the Roles of Information Technology in 
Improving Teaching and Learning in Undergraduate 
Science, Mathematics, Engineering and Technology Education 

June 20-21, 2000 

TUESDAY, JUNE 20, 2000 



8:00 am 


Continental Breakfast 


8:15 am 


Welcome, Introductions, and Overview of Meeting 
Mike Smith, Chair, Workshop Planning Group 


9:00 am 


Session 1: Overview of Current Technologies in Teaching and Learning 
Jean-Pierre Bayard, California State University, Sacramento 

Talk Title: What's Out There : Instructiorial Technology for 
College-Level SM.E&T Instructors 


10:00 am 


Break 


10:15 am 


Ron Stevens, UCLA 

Talk Title: Tracing the Development , Transfer , and Retention 
of Problem-Solving Skills 


11:15 am 


General Discussion 


Noon 


Lunch 
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1:00 pm 


Session 2: The Impact of Technology on Teaching and Learning 
a) What is the impact of current technology on teaching and learning? 

Christopher Dede, George Mason University* 

Talk Title: Using Multiple Interactive Media to Enhance Learning 

Ben Shneiderman, University of Maryland 

Talk Title: Pedagogic Strategies for Applying Educational Technology: 
Relate'Create'Donate 


3:00 pm 


Break 


3:15 pm 


b) Panel Discussion: How can the impact of technology on teaching 
and learning be evaluated most effectively? 

Ron Stevens 
Christopher Dede 
Ben Shneiderman 

Moderator: Mike Smith, Chair, Workshop Planning Group 


5:15 pm 


Summary of the day; plans for Day 2 



WEDNESDAY, JUNE 21, 2000 



8:00 am 


Continental Breakfast 


8:15 am 


Session 3: Thinking about the Future of Information Technologies in 
Teaching and Learning 

How to think about the future; looking at certainties and 
uncertainties, forces and drivers 

Moderator: Alan Schwartz, PolicyFutures 



*At the time of the workshop, Christopher Dede was at George Mason University. He is now at Harvard Uni- _ 



versity. 
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9:00 am Four visions of transformative changes inside and outside the University 
Ted Kahn, President, DesignWorlds for Learning, Inc.; Principal, 
Capital Works, LLC 

Chris Tucker, Chief Strategist, In-Q-Tel 
Philip Agre, Department of Information Studies, 

University of California at Los Angeles 

Richard Larson, Massachusetts Institute of Technology 



10:00 am Break 

10:15 am Discussion of forces and drivers of change, and their implications 
Andries van Dam, Brown University 

12:15 pm Lunch and Workshop Summary 

1:15 pm Adjourn 
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Appendix C 

Workshop Presenters and Participants 



PRESENTERS 
Day 1 

Ronald Stevens, Director, IMMEX Software Development, University of California, Los 
Angeles 

Jean-Pierre Bayard, Department of Electrical &. Electronic Engineering, California State 
University, Sacramento 
Christopher Dede, George Mason University* 

Ben Shneiderman, University of Maryland 



Day 2 

Richard Larson, Massachusetts Institute of Technology 

Ted Kahn, President, DesignWorlds for Learning, Inc.; Principal, CapitalWorks, LLC 
Christopher Tucker, Chief Strategist, In-Q-Tel 



PARTICIPANTS 

Members of the Committee on Information Technology 

Deborah Hughes Hallet, Professor of Mathematics, University of Arizona 
Jack Wilson, Professor of Physics, Rensselaer Polytechnic Institute 

’At the time of the workshop, Christopher Dede was at George Mason University. He is now a professor at 
Harvard University. 
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Stephen Hurst, University of Illinois 
Dorothy Stout, National Science Foundation 



Center for Education Strategic Planning Advisory Group 

Mike Smith, Hewlett Foundation and Stanford University 

Martha Darling, Educational Consultant 

Melvin George, University of Missouri 

Mike Atkin, Stanford University 

Ron Latanision, Massachusetts Institute of Technology 



NRC Staff 



Kevin Aylesworth, Center for Education 
Jay Labov, Center for Education 
Michael Feuer, Center for Education 

Ray Fomes, Government University Industry Research Roundtable 
Terry Holmer, Center for Education 



PolicyFutures 

Alan Schwartz 
Stanley Feder 



Other Participants 

Phil Agre, University of California, Los Angeles 
Lara Couturier, Futures Project 
Andries van Dam, Brown University 

Diane Rogers, Chief of Staff to the Deputy Secretary of the U.S. Department of Education 
Anne Morgan Spalter, Brown University 
James Pellegrino, Vanderbilt University 
Lee Zia, National Science Foundation 
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Appendix D 



Planning Group Biosketches 



Martha Darling is an education policy con- 
sultant in Ann Arbor, Michigan. Prior to 
moving to Ann Arbor, she worked in senior 
management at The Boeing Company in 
Seattle, Washington, from which she is now 
retired. At Boeing, she had assignments 
with the Commercial Airplane Group and 
also served as corporate manager of educa- 
tion affairs. In this capacity, she supported 
the CEO and other executives in their lead- 
ership roles in public education reform at 
the state level and in Seattle. Ms. Darling is 
active in a variety of civic and community 
organizations, serving on the boards of Reed 
College, the National Center for Research 
on Evaluation, Standards and Student Test- 
ing, the White House Fellows Foundation, 
and the Ann Arbor Hands-On Museum. 

Ms. Darling has also participated in a num- 
ber of activities with the National Research 
Council (NRC) of the National Academies. 
She was vice chair of the National Advisory 
Committee for the former NRC Center for 
Science, Mathematics and Engineering 
Education and served as vice chair of the 



advisory board of the NRC Center for Edu- 
cation. 

Deborah Hughes Hallett is Professor of 
Mathematics at the University of Arizona. 
With Andrew M. Gleason at Harvard, she 
organized the Calculus Consortium based at 
Harvard, which brought together faculty 
from a wide variety of schools to improve 
undergraduate education in mathematics. 
With Gleason and other consortium mem- 
bers, Dr. Hughes Hallett has published sev- 
eral popular calculus textbooks that embody 
these improvements. Dr. Hughes Hallett is 
actively involved in discussions about the 
teaching of undergraduate mathematics at 
the national and international level. She is 
a member of the NRC’s Committee on Ad- 
vanced Study in American High Schools 
and of the Mathematical Association of 
America Committee on Mutual Concerns. 
She was chair of the American Women in 
Mathematics group that provided feedback 
to the National Council of Teachers of 
Mathematics when the council developed 
national mathematics education standards. 
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In 1998 she was co-chair of the Interna- 
tional Conference on the Teaching of 
Mathematics in Samos, Greece, attended by 
faculty from 44 countries. Prior to this she 
established programs for mid-career master’s 
students at the Kennedy School of Govern- 
ment, precalculus, and quantitative reason- 
ing courses (with Gleason), and courses for 
economics majors. She was awarded the 
Louise Hay Prize and elected a fellow of the 
American Association for the Advance- 
ment of Science for contributions to math- 
ematics education. 

Marshall S. Smith is education program 
officer with the William and Flora Hewlett 
Foundation, on leave from the School of 
Education at Stanford University. Dr. 

Smith is the former undersecretary and act- 
ing deputy secretary of the U.S. Department 
of Education. He trained originally in sta- 
tistical techniques for research, and acquired 
extensive knowledge of policy issues 
through his years of academic experience. 

He has held key positions in government 
education policy during the 1970’s; re- 
searched topics including computer analysis 
of social science data, early child education, 
critical thinking, and social inequality; and 
taught at Harvard, Wisconsin, and Stanford. 
At Stanford, he was dean of the School of 
Education for six years. With this broad 
background, he is able to integrate research 
on policy questions from several disciplines 
and to focus on educational process, 
whether at the level of the individual stu- 
dent in the classroom or at the level of state 
and national educational reform. Dr. Smith 
is a member of the National Academy of 



Education, Co-Chair of the Pew Forum on 
Education Reform, and Principal Investiga- 
tor of the Pew Network on District Reform. 
He participated in the NRC Committee on 
Information Technology and on the Center 
for Education’s Strategic Planning Advisory 
Group. Dr. Smith received his EdD degree 
in Measurement and Statistics at Harvard 
University in 1970. 

Jack Wilson is professor of management at 
the University of Massachusetts, Amherst, 
and CEO of UMassOnLine, which offers 
University of Massachusetts courses on the 
Internet. Formerly, Wilson was J. Eric 
Jonsson Distinguished Professor of Physics, 
Engineering Science, Information Technol- 
ogy, and Management at Rensselaer Poly- 
technic Institute, and co-director of RPI’s 
Severino Center for Technological Entre- 
preneurship. He founded and was chairman 
of the board of Interactive Learning Inter- 
national (ILINC), and later became chief 
scientist of Mertergy Corporation. He 
joined the faculty of Rensselaer in 1990 and 
moved to the University of Massachusetts in 
2001. Among the many awards he has won, 
Dr. Wilson received the Pew Charitable 
Trusts Leadership Award for Renewal of 
Undergraduate Education in 1996. Dr. 
Wilson has published numerous papers, one 
of which is “Re-engineering the Under- 
graduate Curriculum,” a book chapter for 
The Learning Revolution published by Anker 
Publishing Co. in 1997. Dr. Wilson re- 
ceived his A.B. from Thiel College in 1967, 
and his M.A. and Ph.D. from Kent State 
University in 1972. 
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